2023-12-14T18:06:36.638 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-14T18:06:36.638 DEBUG:teuthology.run:Teuthology command: teuthology --description fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} --name choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi --archive /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435 --owner scheduled_choffman@teuthology --verbose -- /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/orig.config.yaml 2023-12-14T18:06:36.694 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-14T18:06:36.783 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435 branch: wip-choffman-62793 description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} email: null first_in_suite: false job_id: '7491435' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-choffman-62793 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 3 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding sha1: 1faa353368ad4f5aeb56d7107b638336f58659ed subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: 1faa353368ad4f5aeb56d7107b638336f58659ed kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=crc - wsync syntax: v2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-62793-testing sha1: bf026da9d69000f3d4a9d16360156095d6612a2d owner: scheduled_choffman@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 4289 sha1: 1faa353368ad4f5aeb56d7107b638336f58659ed sleep_before_teardown: 0 suite: fs suite_branch: wip-62793-testing suite_path: /home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa suite_relpath: qa suite_repo: https://github.com/chrisphoffman/ceph.git suite_sha1: bf026da9d69000f3d4a9d16360156095d6612a2d syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi093.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0OZ7Zu/Fbt8VVGDDt1T2PvUhU4fESkuZbiUfSgTrPk3/rXRSJoWSk61bsJ8gYyqxF4U5YRHRR4zvtYnG87JB0= smithi097.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInchYnEXB0mnlv+DG8qY+0DrB4YSZgRo8hrpqJg86p8xIVlwQ42sEF39oDtpwBjjvB0oejWS6OZRCTPycQcfN8= smithi160.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNa3qnlFw6f2pdh2OPpaLbJoAkPVdlYJBUYmURBR4LnG+TLB5eIVTAAVAIbzVakwuVbMUo9U2cKgGdoRuPGp2AY= tasks: - pexec: all: - sudo cp /etc/containers/registries.conf /etc/containers/registries.conf.backup - sudo dnf -y module reset container-tools - sudo dnf -y module install container-tools:rhel8 --allowerasing --nobest - sudo cp /etc/containers/registries.conf.backup /etc/containers/registries.conf - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - workunit: clients: all: - fs/misc teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-14_17:59:30 tube: smithi user: choffman verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.232157 2023-12-14T18:06:36.783 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa; will attempt to use it 2023-12-14T18:06:36.784 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks 2023-12-14T18:06:36.784 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-14T18:06:36.786 INFO:teuthology.task.internal:Checking packages... 2023-12-14T18:06:36.809 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '1faa353368ad4f5aeb56d7107b638336f58659ed' 2023-12-14T18:06:36.809 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-14T18:06:36.809 INFO:teuthology.packaging:ref: None 2023-12-14T18:06:36.809 INFO:teuthology.packaging:tag: None 2023-12-14T18:06:36.809 INFO:teuthology.packaging:branch: wip-choffman-62793 2023-12-14T18:06:36.809 INFO:teuthology.packaging:sha1: 1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:06:36.810 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-choffman-62793 2023-12-14T18:06:37.042 INFO:teuthology.task.internal:Found packages for ceph version 18.0.0-3905.g1faa3533 2023-12-14T18:06:37.043 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-14T18:06:37.052 INFO:teuthology.task.internal:no buildpackages task found 2023-12-14T18:06:37.053 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-14T18:06:37.097 INFO:teuthology.task.internal:Saving configuration 2023-12-14T18:06:37.117 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-14T18:06:37.137 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-14T18:06:37.163 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:32.467048', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0OZ7Zu/Fbt8VVGDDt1T2PvUhU4fESkuZbiUfSgTrPk3/rXRSJoWSk61bsJ8gYyqxF4U5YRHRR4zvtYnG87JB0='} 2023-12-14T18:06:37.186 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:32.470154', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInchYnEXB0mnlv+DG8qY+0DrB4YSZgRo8hrpqJg86p8xIVlwQ42sEF39oDtpwBjjvB0oejWS6OZRCTPycQcfN8='} 2023-12-14T18:06:37.205 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:32.468462', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNa3qnlFw6f2pdh2OPpaLbJoAkPVdlYJBUYmURBR4LnG+TLB5eIVTAAVAIbzVakwuVbMUo9U2cKgGdoRuPGp2AY='} 2023-12-14T18:06:37.205 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-14T18:06:37.215 INFO:teuthology.task.internal:roles: ubuntu@smithi093.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2023-12-14T18:06:37.215 INFO:teuthology.task.internal:roles: ubuntu@smithi097.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2023-12-14T18:06:37.215 INFO:teuthology.task.internal:roles: ubuntu@smithi160.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2023-12-14T18:06:37.215 INFO:teuthology.run_tasks:Running task console_log... 2023-12-14T18:06:37.344 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f388f89ddc0>, signals=[15]) 2023-12-14T18:06:37.344 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-14T18:06:37.352 INFO:teuthology.task.internal:Opening connections... 2023-12-14T18:06:37.352 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:06:37.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:06:37.421 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi097.front.sepia.ceph.com 2023-12-14T18:06:37.422 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:06:37.527 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi160.front.sepia.ceph.com 2023-12-14T18:06:37.528 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:06:37.594 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-14T18:06:37.604 DEBUG:teuthology.orchestra.run.smithi093:> uname -m 2023-12-14T18:06:37.624 INFO:teuthology.orchestra.run.smithi093.stdout:x86_64 2023-12-14T18:06:37.624 DEBUG:teuthology.orchestra.run.smithi093:> cat /etc/os-release 2023-12-14T18:06:37.682 INFO:teuthology.orchestra.run.smithi093.stdout:NAME="Red Hat Enterprise Linux" 2023-12-14T18:06:37.682 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION="8.6 (Ootpa)" 2023-12-14T18:06:37.682 INFO:teuthology.orchestra.run.smithi093.stdout:ID="rhel" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:ID_LIKE="fedora" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:VERSION_ID="8.6" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:PLATFORM_ID="platform:el8" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:ANSI_COLOR="0;31" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:HOME_URL="https://www.redhat.com/" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-14T18:06:37.683 INFO:teuthology.orchestra.run.smithi093.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-14T18:06:37.684 INFO:teuthology.lock.ops:Updating smithi093.front.sepia.ceph.com on lock server 2023-12-14T18:06:37.709 DEBUG:teuthology.orchestra.run.smithi097:> uname -m 2023-12-14T18:06:37.731 INFO:teuthology.orchestra.run.smithi097.stdout:x86_64 2023-12-14T18:06:37.731 DEBUG:teuthology.orchestra.run.smithi097:> cat /etc/os-release 2023-12-14T18:06:37.791 INFO:teuthology.orchestra.run.smithi097.stdout:NAME="Red Hat Enterprise Linux" 2023-12-14T18:06:37.791 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION="8.6 (Ootpa)" 2023-12-14T18:06:37.791 INFO:teuthology.orchestra.run.smithi097.stdout:ID="rhel" 2023-12-14T18:06:37.791 INFO:teuthology.orchestra.run.smithi097.stdout:ID_LIKE="fedora" 2023-12-14T18:06:37.791 INFO:teuthology.orchestra.run.smithi097.stdout:VERSION_ID="8.6" 2023-12-14T18:06:37.791 INFO:teuthology.orchestra.run.smithi097.stdout:PLATFORM_ID="platform:el8" 2023-12-14T18:06:37.791 INFO:teuthology.orchestra.run.smithi097.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-14T18:06:37.791 INFO:teuthology.orchestra.run.smithi097.stdout:ANSI_COLOR="0;31" 2023-12-14T18:06:37.792 INFO:teuthology.orchestra.run.smithi097.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-14T18:06:37.792 INFO:teuthology.orchestra.run.smithi097.stdout:HOME_URL="https://www.redhat.com/" 2023-12-14T18:06:37.792 INFO:teuthology.orchestra.run.smithi097.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-14T18:06:37.792 INFO:teuthology.orchestra.run.smithi097.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-14T18:06:37.792 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:06:37.792 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-14T18:06:37.792 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-14T18:06:37.792 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-14T18:06:37.792 INFO:teuthology.orchestra.run.smithi097.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-14T18:06:37.792 INFO:teuthology.lock.ops:Updating smithi097.front.sepia.ceph.com on lock server 2023-12-14T18:06:37.818 DEBUG:teuthology.orchestra.run.smithi160:> uname -m 2023-12-14T18:06:37.838 INFO:teuthology.orchestra.run.smithi160.stdout:x86_64 2023-12-14T18:06:37.838 DEBUG:teuthology.orchestra.run.smithi160:> cat /etc/os-release 2023-12-14T18:06:37.898 INFO:teuthology.orchestra.run.smithi160.stdout:NAME="Red Hat Enterprise Linux" 2023-12-14T18:06:37.898 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION="8.6 (Ootpa)" 2023-12-14T18:06:37.898 INFO:teuthology.orchestra.run.smithi160.stdout:ID="rhel" 2023-12-14T18:06:37.898 INFO:teuthology.orchestra.run.smithi160.stdout:ID_LIKE="fedora" 2023-12-14T18:06:37.898 INFO:teuthology.orchestra.run.smithi160.stdout:VERSION_ID="8.6" 2023-12-14T18:06:37.898 INFO:teuthology.orchestra.run.smithi160.stdout:PLATFORM_ID="platform:el8" 2023-12-14T18:06:37.898 INFO:teuthology.orchestra.run.smithi160.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2023-12-14T18:06:37.898 INFO:teuthology.orchestra.run.smithi160.stdout:ANSI_COLOR="0;31" 2023-12-14T18:06:37.898 INFO:teuthology.orchestra.run.smithi160.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2023-12-14T18:06:37.899 INFO:teuthology.orchestra.run.smithi160.stdout:HOME_URL="https://www.redhat.com/" 2023-12-14T18:06:37.899 INFO:teuthology.orchestra.run.smithi160.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2023-12-14T18:06:37.899 INFO:teuthology.orchestra.run.smithi160.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-12-14T18:06:37.899 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:06:37.899 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2023-12-14T18:06:37.899 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2023-12-14T18:06:37.899 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2023-12-14T18:06:37.899 INFO:teuthology.orchestra.run.smithi160.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2023-12-14T18:06:37.899 INFO:teuthology.lock.ops:Updating smithi160.front.sepia.ceph.com on lock server 2023-12-14T18:06:37.918 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-14T18:06:37.930 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-14T18:06:37.957 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-14T18:06:37.957 DEBUG:teuthology.orchestra.run.smithi093:> test '!' -e /home/ubuntu/cephtest 2023-12-14T18:06:37.959 DEBUG:teuthology.orchestra.run.smithi097:> test '!' -e /home/ubuntu/cephtest 2023-12-14T18:06:37.962 DEBUG:teuthology.orchestra.run.smithi160:> test '!' -e /home/ubuntu/cephtest 2023-12-14T18:06:37.980 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-14T18:06:38.002 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-14T18:06:38.002 DEBUG:teuthology.orchestra.run.smithi093:> test -z $(ls -A /var/lib/ceph) 2023-12-14T18:06:38.019 DEBUG:teuthology.orchestra.run.smithi097:> test -z $(ls -A /var/lib/ceph) 2023-12-14T18:06:38.021 DEBUG:teuthology.orchestra.run.smithi160:> test -z $(ls -A /var/lib/ceph) 2023-12-14T18:06:38.047 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-14T18:06:38.125 INFO:teuthology.run_tasks:Running task kernel... 2023-12-14T18:06:38.142 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2023-12-14T18:06:38.142 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2023-12-14T18:06:38.142 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2023-12-14T18:06:38.143 DEBUG:teuthology.orchestra.run.smithi093:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-14T18:06:38.161 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:38.162 DEBUG:teuthology.orchestra.run.smithi093:> uname -r 2023-12-14T18:06:38.222 INFO:teuthology.orchestra.run.smithi093.stdout:4.18.0-513.9.1.el8_9.x86_64 2023-12-14T18:06:38.222 INFO:teuthology.task.kernel:Running kernel on smithi093: 4.18.0-513.9.1.el8_9.x86_64 2023-12-14T18:06:38.222 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum install -y kernel 2023-12-14T18:06:39.947 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-12-14T18:06:39.951 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2023-12-14T18:06:40.091 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:06:40.092 INFO:teuthology.orchestra.run.smithi093.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-14T18:06:40.092 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:06:41.891 INFO:teuthology.orchestra.run.smithi093.stdout:Copr repo for python3-asyncssh owned by ceph 15 kB/s | 3.5 kB 00:00 2023-12-14T18:06:43.969 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 7.8 MB/s | 16 MB 00:02 2023-12-14T18:06:47.516 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 528 kB/s | 24 kB 00:00 2023-12-14T18:06:48.294 INFO:teuthology.orchestra.run.smithi093.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2023-12-14T18:06:48.294 INFO:teuthology.orchestra.run.smithi093.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2023-12-14T18:06:48.367 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-12-14T18:06:48.368 INFO:teuthology.orchestra.run.smithi093.stdout:Nothing to do. 2023-12-14T18:06:48.368 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-12-14T18:06:48.525 DEBUG:teuthology.orchestra.run.smithi093:> echo no | sudo yum reinstall kernel || true 2023-12-14T18:06:48.902 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-12-14T18:06:48.902 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2023-12-14T18:06:48.967 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:06:48.967 INFO:teuthology.orchestra.run.smithi093.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-14T18:06:48.967 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:06:49.271 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:00:02 ago on Thu 14 Dec 2023 06:06:47 PM UTC. 2023-12-14T18:06:49.351 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-12-14T18:06:49.351 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-14T18:06:49.388 INFO:teuthology.orchestra.run.smithi093.stderr:Error: No packages marked for reinstall. 2023-12-14T18:06:49.435 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum reinstall -y kernel || true 2023-12-14T18:06:49.770 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-12-14T18:06:49.770 INFO:teuthology.orchestra.run.smithi093.stdout:Unable to read consumer identity 2023-12-14T18:06:49.825 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:06:49.825 INFO:teuthology.orchestra.run.smithi093.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2023-12-14T18:06:49.825 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:06:50.131 INFO:teuthology.orchestra.run.smithi093.stdout:Last metadata expiration check: 0:00:03 ago on Thu 14 Dec 2023 06:06:47 PM UTC. 2023-12-14T18:06:50.195 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2023-12-14T18:06:50.195 INFO:teuthology.orchestra.run.smithi093.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2023-12-14T18:06:50.236 INFO:teuthology.orchestra.run.smithi093.stderr:Error: No packages marked for reinstall. 2023-12-14T18:06:50.288 DEBUG:teuthology.orchestra.run.smithi093:> rpm -q kernel | sort -rV | head -n 1 2023-12-14T18:06:50.458 INFO:teuthology.orchestra.run.smithi093.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2023-12-14T18:06:50.458 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2023-12-14T18:06:50.458 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2023-12-14T18:06:50.458 DEBUG:teuthology.parallel:result is None 2023-12-14T18:06:50.458 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-14T18:06:50.467 INFO:teuthology.task.internal:Creating test directory... 2023-12-14T18:06:50.467 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-14T18:06:50.469 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-14T18:06:50.472 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-14T18:06:50.492 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-14T18:06:50.503 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-14T18:06:50.530 INFO:teuthology.task.internal:Creating archive directory... 2023-12-14T18:06:50.530 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-14T18:06:50.532 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-14T18:06:50.535 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-14T18:06:50.577 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-14T18:06:50.584 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-14T18:06:50.585 DEBUG:teuthology.orchestra.run.smithi097:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-14T18:06:50.629 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:50.630 DEBUG:teuthology.orchestra.run.smithi160:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-14T18:06:50.648 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:50.649 DEBUG:teuthology.orchestra.run.smithi093:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-14T18:06:50.651 DEBUG:teuthology.orchestra.run.smithi097:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-14T18:06:50.673 DEBUG:teuthology.orchestra.run.smithi160:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-14T18:06:50.688 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:50.706 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:50.716 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:50.728 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:50.736 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:50.744 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-14T18:06:50.745 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-14T18:06:50.905 INFO:teuthology.task.internal:Configuring sudo... 2023-12-14T18:06:50.905 DEBUG:teuthology.orchestra.run.smithi093:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-14T18:06:50.907 DEBUG:teuthology.orchestra.run.smithi097:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-14T18:06:50.910 DEBUG:teuthology.orchestra.run.smithi160:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-14T18:06:50.940 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-14T18:06:50.951 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-14T18:06:50.952 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-14T18:06:50.975 DEBUG:teuthology.orchestra.run.smithi097:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-14T18:06:50.980 DEBUG:teuthology.orchestra.run.smithi160:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-14T18:06:50.999 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:51.073 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:51.133 DEBUG:teuthology.orchestra.run.smithi093:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:51.217 DEBUG:teuthology.orchestra.run.smithi093:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:51.258 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:06:51.258 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-14T18:06:51.324 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:51.370 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:51.428 DEBUG:teuthology.orchestra.run.smithi097:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:51.515 DEBUG:teuthology.orchestra.run.smithi097:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:51.558 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:06:51.558 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-14T18:06:51.627 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:51.675 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-14T18:06:51.739 DEBUG:teuthology.orchestra.run.smithi160:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:51.781 DEBUG:teuthology.orchestra.run.smithi160:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-14T18:06:51.828 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:06:51.828 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-14T18:06:51.896 DEBUG:teuthology.orchestra.run.smithi093:> sudo service rsyslog restart 2023-12-14T18:06:51.898 DEBUG:teuthology.orchestra.run.smithi097:> sudo service rsyslog restart 2023-12-14T18:06:51.900 DEBUG:teuthology.orchestra.run.smithi160:> sudo service rsyslog restart 2023-12-14T18:06:51.952 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:06:51.956 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:06:51.991 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:06:52.509 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-14T18:06:52.516 INFO:teuthology.task.internal:Starting timer... 2023-12-14T18:06:52.516 INFO:teuthology.run_tasks:Running task pcp... 2023-12-14T18:06:52.543 INFO:teuthology.run_tasks:Running task selinux... 2023-12-14T18:06:52.579 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:logrotate_t:s0']} 2023-12-14T18:06:52.579 DEBUG:teuthology.orchestra.run.smithi093:> sudo service auditd rotate 2023-12-14T18:06:52.648 INFO:teuthology.orchestra.run.smithi093.stdout:Rotating logs: 2023-12-14T18:06:52.650 DEBUG:teuthology.orchestra.run.smithi097:> sudo service auditd rotate 2023-12-14T18:06:52.718 INFO:teuthology.orchestra.run.smithi097.stdout:Rotating logs: 2023-12-14T18:06:52.719 DEBUG:teuthology.orchestra.run.smithi160:> sudo service auditd rotate 2023-12-14T18:06:52.793 INFO:teuthology.orchestra.run.smithi160.stdout:Rotating logs: 2023-12-14T18:06:52.794 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-14T18:06:52.794 DEBUG:teuthology.orchestra.run.smithi093:> /usr/sbin/getenforce 2023-12-14T18:06:52.818 INFO:teuthology.orchestra.run.smithi093.stdout:Permissive 2023-12-14T18:06:52.818 DEBUG:teuthology.orchestra.run.smithi097:> /usr/sbin/getenforce 2023-12-14T18:06:52.854 INFO:teuthology.orchestra.run.smithi097.stdout:Permissive 2023-12-14T18:06:52.854 DEBUG:teuthology.orchestra.run.smithi160:> /usr/sbin/getenforce 2023-12-14T18:06:52.884 INFO:teuthology.orchestra.run.smithi160.stdout:Permissive 2023-12-14T18:06:52.884 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi093.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi097.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi160.front.sepia.ceph.com': 'permissive'} 2023-12-14T18:06:52.884 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:06:52.910 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:52.911 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:06:52.937 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:52.938 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:06:52.965 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:06:52.965 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-14T18:06:52.965 DEBUG:teuthology.orchestra.run.smithi093:> sudo /usr/sbin/setenforce permissive 2023-12-14T18:06:52.992 DEBUG:teuthology.orchestra.run.smithi097:> sudo /usr/sbin/setenforce permissive 2023-12-14T18:06:53.019 DEBUG:teuthology.orchestra.run.smithi160:> sudo /usr/sbin/setenforce permissive 2023-12-14T18:06:53.045 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-14T18:06:53.054 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-14T18:06:53.054 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-14T18:06:53.071 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-14T18:06:53.072 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi093.front.sepia.ceph.com,smithi097.front.sepia.ceph.com,smithi160.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-14T18:14:03.775 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi093.front.sepia.ceph.com'), Remote(name='ubuntu@smithi097.front.sepia.ceph.com'), Remote(name='ubuntu@smithi160.front.sepia.ceph.com')] 2023-12-14T18:14:03.776 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-12-14T18:14:03.777 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi093.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:14:03.847 DEBUG:teuthology.orchestra.run.smithi093:> true 2023-12-14T18:14:03.872 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi093.front.sepia.ceph.com' 2023-12-14T18:14:03.872 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-12-14T18:14:03.873 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi097.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:14:03.938 DEBUG:teuthology.orchestra.run.smithi097:> true 2023-12-14T18:14:04.007 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi097.front.sepia.ceph.com' 2023-12-14T18:14:04.008 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-12-14T18:14:04.008 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi160.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-14T18:14:04.075 DEBUG:teuthology.orchestra.run.smithi160:> true 2023-12-14T18:14:04.147 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi160.front.sepia.ceph.com' 2023-12-14T18:14:04.147 INFO:teuthology.run_tasks:Running task clock... 2023-12-14T18:14:04.159 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-14T18:14:04.160 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-14T18:14:04.160 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:14:04.162 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-14T18:14:04.162 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:14:04.164 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-14T18:14:04.164 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:14:04.193 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-14T18:14:04.195 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-14T18:14:04.208 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-14T18:14:04.210 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-14T18:14:04.220 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-12-14T18:14:04.235 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-12-14T18:14:04.276 INFO:teuthology.orchestra.run.smithi097.stderr:sudo: ntpd: command not found 2023-12-14T18:14:04.279 INFO:teuthology.orchestra.run.smithi093.stderr:sudo: ntpd: command not found 2023-12-14T18:14:04.288 INFO:teuthology.orchestra.run.smithi097.stdout:506 Cannot talk to daemon 2023-12-14T18:14:04.291 INFO:teuthology.orchestra.run.smithi093.stdout:506 Cannot talk to daemon 2023-12-14T18:14:04.303 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-14T18:14:04.306 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-14T18:14:04.308 INFO:teuthology.orchestra.run.smithi160.stderr:sudo: ntpd: command not found 2023-12-14T18:14:04.317 INFO:teuthology.orchestra.run.smithi097.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-14T18:14:04.320 INFO:teuthology.orchestra.run.smithi093.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-14T18:14:04.320 INFO:teuthology.orchestra.run.smithi160.stdout:506 Cannot talk to daemon 2023-12-14T18:14:04.335 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-12-14T18:14:04.349 INFO:teuthology.orchestra.run.smithi160.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-12-14T18:14:04.387 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-12-14T18:14:04.388 INFO:teuthology.orchestra.run.smithi093.stderr:bash: ntpq: command not found 2023-12-14T18:14:04.390 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:14:04.391 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-12-14T18:14:04.391 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.391 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.391 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.391 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.391 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:14:04.392 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-12-14T18:14:04.392 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.392 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.392 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.392 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.419 INFO:teuthology.orchestra.run.smithi160.stderr:bash: ntpq: command not found 2023-12-14T18:14:04.423 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:14:04.423 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2023-12-14T18:14:04.423 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.423 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.423 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.423 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:14:04.424 INFO:teuthology.run_tasks:Running task pexec... 2023-12-14T18:14:04.435 INFO:teuthology.task.pexec:Executing custom commands... 2023-12-14T18:14:04.436 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:14:04.436 DEBUG:teuthology.orchestra.run.smithi093:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-14T18:14:04.436 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi097.front.sepia.ceph.com 2023-12-14T18:14:04.436 DEBUG:teuthology.orchestra.run.smithi097:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-14T18:14:04.437 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi160.front.sepia.ceph.com 2023-12-14T18:14:04.437 DEBUG:teuthology.orchestra.run.smithi160:> TESTDIR=/home/ubuntu/cephtest bash -s 2023-12-14T18:14:04.812 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:04.813 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:04.844 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:06.874 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2023-12-14T18:14:06.966 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2023-12-14T18:14:07.039 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2023-12-14T18:14:07.187 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 41 kB/s | 4.5 kB 00:00 2023-12-14T18:14:07.619 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2023-12-14T18:14:07.659 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 24 kB/s | 4.1 kB 00:00 2023-12-14T18:14:10.155 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-12-14T18:14:10.156 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-12-14T18:14:10.156 INFO:teuthology.orchestra.run.smithi093.stdout: Package Architecture Version Repository Size 2023-12-14T18:14:10.156 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-12-14T18:14:10.156 INFO:teuthology.orchestra.run.smithi093.stdout:Resetting modules: 2023-12-14T18:14:10.156 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools 2023-12-14T18:14:10.156 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:14:10.156 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-12-14T18:14:10.156 INFO:teuthology.orchestra.run.smithi093.stdout:================================================================================ 2023-12-14T18:14:10.157 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:14:10.389 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-12-14T18:14:10.390 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-12-14T18:14:10.390 INFO:teuthology.orchestra.run.smithi097.stdout: Package Architecture Version Repository Size 2023-12-14T18:14:10.391 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-12-14T18:14:10.391 INFO:teuthology.orchestra.run.smithi097.stdout:Resetting modules: 2023-12-14T18:14:10.391 INFO:teuthology.orchestra.run.smithi097.stdout: container-tools 2023-12-14T18:14:10.391 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:14:10.391 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-12-14T18:14:10.391 INFO:teuthology.orchestra.run.smithi097.stdout:================================================================================ 2023-12-14T18:14:10.391 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:14:10.554 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2023-12-14T18:14:10.555 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2023-12-14T18:14:10.555 INFO:teuthology.orchestra.run.smithi160.stdout: Package Architecture Version Repository Size 2023-12-14T18:14:10.556 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2023-12-14T18:14:10.556 INFO:teuthology.orchestra.run.smithi160.stdout:Resetting modules: 2023-12-14T18:14:10.556 INFO:teuthology.orchestra.run.smithi160.stdout: container-tools 2023-12-14T18:14:10.556 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:14:10.556 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2023-12-14T18:14:10.556 INFO:teuthology.orchestra.run.smithi160.stdout:================================================================================ 2023-12-14T18:14:10.556 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:14:20.696 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-12-14T18:14:21.148 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:23.068 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2023-12-14T18:14:23.468 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-12-14T18:14:23.478 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2023-12-14T18:14:23.495 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 22 kB/s | 4.5 kB 00:00 2023-12-14T18:14:23.946 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:23.959 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:25.685 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 kB/s | 4.5 kB 00:00 2023-12-14T18:14:26.054 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 21 kB/s | 4.5 kB 00:00 2023-12-14T18:14:26.437 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 18 kB/s | 4.1 kB 00:00 2023-12-14T18:14:26.471 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-12-14T18:14:26.625 INFO:teuthology.orchestra.run.smithi093.stdout:========================================================================================================== 2023-12-14T18:14:26.625 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-12-14T18:14:26.625 INFO:teuthology.orchestra.run.smithi093.stdout:========================================================================================================== 2023-12-14T18:14:26.625 INFO:teuthology.orchestra.run.smithi093.stdout:Installing group/module packages: 2023-12-14T18:14:26.625 INFO:teuthology.orchestra.run.smithi093.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2023-12-14T18:14:26.625 INFO:teuthology.orchestra.run.smithi093.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2023-12-14T18:14:26.625 INFO:teuthology.orchestra.run.smithi093.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2023-12-14T18:14:26.625 INFO:teuthology.orchestra.run.smithi093.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-14T18:14:26.626 INFO:teuthology.orchestra.run.smithi093.stdout:Installing weak dependencies: 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout:Installing module profiles: 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools/common 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling module streams: 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout: container-tools rhel8 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout:========================================================================================================== 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout:Install 13 Packages 2023-12-14T18:14:26.627 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:14:26.628 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 26 M 2023-12-14T18:14:26.628 INFO:teuthology.orchestra.run.smithi093.stdout:Installed size: 87 M 2023-12-14T18:14:26.628 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-12-14T18:14:26.629 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 36 kB/s | 4.1 kB 00:00 2023-12-14T18:14:26.810 INFO:teuthology.orchestra.run.smithi093.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 136 kB/s | 25 kB 00:00 2023-12-14T18:14:26.835 INFO:teuthology.orchestra.run.smithi093.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 453 kB/s | 94 kB 00:00 2023-12-14T18:14:26.887 INFO:teuthology.orchestra.run.smithi093.stdout:(3/13): yajl-2.1.0-12.el8.x86_64.rpm 158 kB/s | 41 kB 00:00 2023-12-14T18:14:26.927 INFO:teuthology.orchestra.run.smithi093.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 8.6 MB/s | 1.0 MB 00:00 2023-12-14T18:14:26.961 INFO:teuthology.orchestra.run.smithi093.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 9.9 MB/s | 739 kB 00:00 2023-12-14T18:14:27.161 INFO:teuthology.orchestra.run.smithi093.stdout:(6/13): buildah-1.31.3-1.module+el8.9.0+20326+3 27 MB/s | 8.8 MB 00:00 2023-12-14T18:14:27.186 INFO:teuthology.orchestra.run.smithi093.stdout:(7/13): criu-libs-3.18-4.module+el8.9.0+20326+3 152 kB/s | 39 kB 00:00 2023-12-14T18:14:27.212 INFO:teuthology.orchestra.run.smithi093.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+20326+38708 955 kB/s | 239 kB 00:00 2023-12-14T18:14:27.320 INFO:teuthology.orchestra.run.smithi093.stdout:(9/13): python3-podman-4.6.0-1.module+el8.9.0+2 1.1 MB/s | 153 kB 00:00 2023-12-14T18:14:27.420 INFO:teuthology.orchestra.run.smithi093.stdout:(10/13): netavark-1.7.0-1.module+el8.9.0+20326+ 14 MB/s | 3.7 MB 00:00 2023-12-14T18:14:27.504 INFO:teuthology.orchestra.run.smithi093.stdout:(11/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 14 MB/s | 2.5 MB 00:00 2023-12-14T18:14:27.529 INFO:teuthology.orchestra.run.smithi093.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+20326+38 456 kB/s | 49 kB 00:00 2023-12-14T18:14:27.672 INFO:teuthology.orchestra.run.smithi093.stdout:(13/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 18 MB/s | 8.1 MB 00:00 2023-12-14T18:14:27.672 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:14:27.672 INFO:teuthology.orchestra.run.smithi093.stdout:Total 24 MB/s | 26 MB 00:01 2023-12-14T18:14:27.947 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-12-14T18:14:27.969 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-12-14T18:14:27.969 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-12-14T18:14:28.581 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-12-14T18:14:28.590 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-12-14T18:14:29.240 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-12-14T18:14:29.317 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-12-14T18:14:29.341 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2023-12-14T18:14:29.467 INFO:teuthology.orchestra.run.smithi097.stdout:========================================================================================================== 2023-12-14T18:14:29.467 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2023-12-14T18:14:29.467 INFO:teuthology.orchestra.run.smithi097.stdout:========================================================================================================== 2023-12-14T18:14:29.467 INFO:teuthology.orchestra.run.smithi097.stdout:Installing group/module packages: 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout:Installing weak dependencies: 2023-12-14T18:14:29.468 INFO:teuthology.orchestra.run.smithi097.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout:Installing module profiles: 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout: container-tools/common 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling module streams: 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout: container-tools rhel8 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout:========================================================================================================== 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout:Install 13 Packages 2023-12-14T18:14:29.469 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:14:29.470 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 26 M 2023-12-14T18:14:29.470 INFO:teuthology.orchestra.run.smithi097.stdout:Installed size: 87 M 2023-12-14T18:14:29.470 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2023-12-14T18:14:29.492 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================================== 2023-12-14T18:14:29.492 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2023-12-14T18:14:29.492 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================================== 2023-12-14T18:14:29.492 INFO:teuthology.orchestra.run.smithi160.stdout:Installing group/module packages: 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: aardvark-dns x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 1.0 M 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: buildah x86_64 1:1.31.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.8 M 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: cockpit-podman noarch 75-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 739 k 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: crun x86_64 1.8.7-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 239 k 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: netavark x86_64 2:1.7.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: python3-podman noarch 4.6.0-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 153 k 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: skopeo x86_64 2:1.13.3-1.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 8.1 M 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: toolbox x86_64 0.0.99.4-5.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 2.5 M 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: udica noarch 0.2.6-20.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 49 k 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytoml noarch 0.1.14-5.git7dea353.el8 rhel-8-for-x86_64-appstream-rpms 25 k 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyxdg noarch 0.25-16.el8 rhel-8-for-x86_64-appstream-rpms 94 k 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout: yajl x86_64 2.1.0-12.el8 rhel-8-for-x86_64-appstream-rpms 41 k 2023-12-14T18:14:29.493 INFO:teuthology.orchestra.run.smithi160.stdout:Installing weak dependencies: 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: criu-libs x86_64 3.18-4.module+el8.9.0+20326+387084d0 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout:Installing module profiles: 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: container-tools/common 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling module streams: 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: container-tools rhel8 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout:========================================================================================================== 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout:Install 13 Packages 2023-12-14T18:14:29.494 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:14:29.495 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 26 M 2023-12-14T18:14:29.495 INFO:teuthology.orchestra.run.smithi160.stdout:Installed size: 87 M 2023-12-14T18:14:29.495 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2023-12-14T18:14:29.509 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2023-12-14T18:14:29.617 INFO:teuthology.orchestra.run.smithi160.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 204 kB/s | 25 kB 00:00 2023-12-14T18:14:29.620 INFO:teuthology.orchestra.run.smithi097.stdout:(1/13): python3-pytoml-0.1.14-5.git7dea353.el8. 165 kB/s | 25 kB 00:00 2023-12-14T18:14:29.654 INFO:teuthology.orchestra.run.smithi097.stdout:(2/13): yajl-2.1.0-12.el8.x86_64.rpm 223 kB/s | 41 kB 00:00 2023-12-14T18:14:29.659 INFO:teuthology.orchestra.run.smithi160.stdout:(2/13): python3-pyxdg-0.25-16.el8.noarch.rpm 574 kB/s | 94 kB 00:00 2023-12-14T18:14:29.679 INFO:teuthology.orchestra.run.smithi097.stdout:(3/13): python3-pyxdg-0.25-16.el8.noarch.rpm 450 kB/s | 94 kB 00:00 2023-12-14T18:14:29.718 INFO:teuthology.orchestra.run.smithi160.stdout:(3/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 10 MB/s | 1.0 MB 00:00 2023-12-14T18:14:29.738 INFO:teuthology.orchestra.run.smithi097.stdout:(4/13): aardvark-dns-1.7.0-1.module+el8.9.0+203 8.6 MB/s | 1.0 MB 00:00 2023-12-14T18:14:29.743 INFO:teuthology.orchestra.run.smithi160.stdout:(4/13): yajl-2.1.0-12.el8.x86_64.rpm 166 kB/s | 41 kB 00:00 2023-12-14T18:14:29.847 INFO:teuthology.orchestra.run.smithi160.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 5.6 MB/s | 739 kB 00:00 2023-12-14T18:14:29.859 INFO:teuthology.orchestra.run.smithi097.stdout:(5/13): cockpit-podman-75-1.module+el8.9.0+2032 4.0 MB/s | 739 kB 00:00 2023-12-14T18:14:29.877 INFO:teuthology.orchestra.run.smithi160.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 295 kB/s | 39 kB 00:00 2023-12-14T18:14:29.922 INFO:teuthology.orchestra.run.smithi097.stdout:(6/13): criu-libs-3.18-4.module+el8.9.0+20326+3 214 kB/s | 39 kB 00:00 2023-12-14T18:14:29.994 INFO:teuthology.orchestra.run.smithi160.stdout:(7/13): buildah-1.31.3-1.module+el8.9.0+20326+3 26 MB/s | 8.8 MB 00:00 2023-12-14T18:14:30.019 INFO:teuthology.orchestra.run.smithi160.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+20326+38708 1.4 MB/s | 239 kB 00:00 2023-12-14T18:14:30.030 INFO:teuthology.orchestra.run.smithi097.stdout:(7/13): buildah-1.31.3-1.module+el8.9.0+20326+3 23 MB/s | 8.8 MB 00:00 2023-12-14T18:14:30.045 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2023-12-14T18:14:30.056 INFO:teuthology.orchestra.run.smithi097.stdout:(8/13): crun-1.8.7-1.module+el8.9.0+20326+38708 1.2 MB/s | 239 kB 00:00 2023-12-14T18:14:30.125 INFO:teuthology.orchestra.run.smithi160.stdout:(9/13): netavark-1.7.0-1.module+el8.9.0+20326+3 15 MB/s | 3.7 MB 00:00 2023-12-14T18:14:30.163 INFO:teuthology.orchestra.run.smithi097.stdout:(9/13): netavark-1.7.0-1.module+el8.9.0+20326+3 15 MB/s | 3.7 MB 00:00 2023-12-14T18:14:30.203 INFO:teuthology.orchestra.run.smithi160.stdout:(10/13): python3-podman-4.6.0-1.module+el8.9.0+ 735 kB/s | 153 kB 00:00 2023-12-14T18:14:30.264 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2023-12-14T18:14:30.264 INFO:teuthology.orchestra.run.smithi097.stdout:(10/13): python3-podman-4.6.0-1.module+el8.9.0+ 656 kB/s | 153 kB 00:00 2023-12-14T18:14:30.357 INFO:teuthology.orchestra.run.smithi097.stdout:(11/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 27 MB/s | 8.1 MB 00:00 2023-12-14T18:14:30.362 INFO:teuthology.orchestra.run.smithi160.stdout:(11/13): skopeo-1.13.3-1.module+el8.9.0+20326+3 24 MB/s | 8.1 MB 00:00 2023-12-14T18:14:30.411 INFO:teuthology.orchestra.run.smithi160.stdout:(12/13): udica-0.2.6-20.module+el8.9.0+20326+38 238 kB/s | 49 kB 00:00 2023-12-14T18:14:30.432 INFO:teuthology.orchestra.run.smithi097.stdout:(12/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 9.4 MB/s | 2.5 MB 00:00 2023-12-14T18:14:30.466 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2023-12-14T18:14:30.470 INFO:teuthology.orchestra.run.smithi160.stdout:(13/13): toolbox-0.0.99.4-5.module+el8.9.0+2032 7.3 MB/s | 2.5 MB 00:00 2023-12-14T18:14:30.471 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:14:30.471 INFO:teuthology.orchestra.run.smithi160.stdout:Total 26 MB/s | 26 MB 00:00 2023-12-14T18:14:30.482 INFO:teuthology.orchestra.run.smithi097.stdout:(13/13): udica-0.2.6-20.module+el8.9.0+20326+38 228 kB/s | 49 kB 00:00 2023-12-14T18:14:30.482 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:14:30.482 INFO:teuthology.orchestra.run.smithi097.stdout:Total 25 MB/s | 26 MB 00:01 2023-12-14T18:14:30.669 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2023-12-14T18:14:30.749 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2023-12-14T18:14:30.761 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2023-12-14T18:14:30.770 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2023-12-14T18:14:30.770 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2023-12-14T18:14:30.782 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2023-12-14T18:14:30.782 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2023-12-14T18:14:30.929 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2023-12-14T18:14:31.195 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2023-12-14T18:14:31.434 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2023-12-14T18:14:31.443 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2023-12-14T18:14:31.455 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2023-12-14T18:14:31.464 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2023-12-14T18:14:32.097 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2023-12-14T18:14:32.113 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:14:32.116 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2023-12-14T18:14:32.343 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2023-12-14T18:14:32.363 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 1/13 2023-12-14T18:14:32.394 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:14:32.817 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2023-12-14T18:14:32.864 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2023-12-14T18:14:32.875 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 2/13 2023-12-14T18:14:33.070 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2023-12-14T18:14:33.085 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 3/13 2023-12-14T18:14:33.273 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2023-12-14T18:14:33.295 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : yajl-2.1.0-12.el8.x86_64 4/13 2023-12-14T18:14:33.458 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2023-12-14T18:14:33.477 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyxdg-0.25-16.el8.noarch 5/13 2023-12-14T18:14:33.694 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2023-12-14T18:14:33.739 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2023-12-14T18:14:33.740 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 6/13 2023-12-14T18:14:33.923 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2023-12-14T18:14:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2023-12-14T18:14:33.996 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 7/13 2023-12-14T18:14:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2023-12-14T18:14:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2023-12-14T18:14:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2023-12-14T18:14:34.895 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:14:34.932 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:14:35.146 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:14:35.198 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:14:35.574 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2023-12-14T18:14:35.627 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 10/13 2023-12-14T18:14:36.504 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2023-12-14T18:14:36.542 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 11/13 2023-12-14T18:14:36.770 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2023-12-14T18:14:36.796 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 12/13 2023-12-14T18:14:36.984 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:14:37.006 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:14:37.410 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2023-12-14T18:14:37.411 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2023-12-14T18:14:37.434 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 13/13 2023-12-14T18:14:37.434 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytoml-0.1.14-5.git7dea353.el8.noarch 1/13 2023-12-14T18:14:37.434 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyxdg-0.25-16.el8.noarch 2/13 2023-12-14T18:14:37.434 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : yajl-2.1.0-12.el8.x86_64 3/13 2023-12-14T18:14:37.434 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084 4/13 2023-12-14T18:14:37.435 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x 5/13 2023-12-14T18:14:37.435 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cockpit-podman-75-1.module+el8.9.0+20326+387084d0. 6/13 2023-12-14T18:14:37.435 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86 7/13 2023-12-14T18:14:37.435 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 8/13 2023-12-14T18:14:37.435 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x 9/13 2023-12-14T18:14:37.435 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-podman-4.6.0-1.module+el8.9.0+20326+387084 10/13 2023-12-14T18:14:37.435 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x8 11/13 2023-12-14T18:14:37.435 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x 12/13 2023-12-14T18:14:45.327 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2023-12-14T18:14:45.327 INFO:teuthology.orchestra.run.smithi093.stdout:Installed products updated. 2023-12-14T18:14:45.873 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:14:45.873 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2023-12-14T18:14:45.873 INFO:teuthology.orchestra.run.smithi093.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:45.873 INFO:teuthology.orchestra.run.smithi093.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:45.873 INFO:teuthology.orchestra.run.smithi093.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:14:45.873 INFO:teuthology.orchestra.run.smithi093.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:45.873 INFO:teuthology.orchestra.run.smithi093.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:14:45.874 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-12-14T18:14:46.043 DEBUG:teuthology.parallel:result is None 2023-12-14T18:14:51.070 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2023-12-14T18:14:51.070 INFO:teuthology.orchestra.run.smithi160.stdout:Installed products updated. 2023-12-14T18:14:51.607 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:14:51.607 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2023-12-14T18:14:51.607 INFO:teuthology.orchestra.run.smithi160.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:51.607 INFO:teuthology.orchestra.run.smithi160.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:51.607 INFO:teuthology.orchestra.run.smithi160.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:14:51.607 INFO:teuthology.orchestra.run.smithi160.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:51.607 INFO:teuthology.orchestra.run.smithi160.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:14:51.608 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2023-12-14T18:14:51.780 DEBUG:teuthology.parallel:result is None 2023-12-14T18:14:52.247 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarc 13/13 2023-12-14T18:14:52.247 INFO:teuthology.orchestra.run.smithi097.stdout:Installed products updated. 2023-12-14T18:14:52.779 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: aardvark-dns-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: buildah-1:1.31.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: cockpit-podman-75-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: criu-libs-3.18-4.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: crun-1.8.7-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: netavark-2:1.7.0-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: python3-podman-4.6.0-1.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytoml-0.1.14-5.git7dea353.el8.noarch 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyxdg-0.25-16.el8.noarch 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: skopeo-2:1.13.3-1.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: toolbox-0.0.99.4-5.module+el8.9.0+20326+387084d0.x86_64 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: udica-0.2.6-20.module+el8.9.0+20326+387084d0.noarch 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: yajl-2.1.0-12.el8.x86_64 2023-12-14T18:14:52.780 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:14:52.781 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-12-14T18:14:52.951 DEBUG:teuthology.parallel:result is None 2023-12-14T18:14:52.952 INFO:teuthology.run_tasks:Running task install... 2023-12-14T18:14:52.961 DEBUG:teuthology.task.install:project ceph 2023-12-14T18:14:52.961 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed'}} 2023-12-14T18:14:52.961 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed'} 2023-12-14T18:14:52.962 INFO:teuthology.task.install:Using flavor: default 2023-12-14T18:14:52.968 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-14T18:14:52.969 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-12-14T18:14:52.969 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'tag': None, 'wait_for_package': False} 2023-12-14T18:14:52.970 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:14:52.972 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'tag': None, 'wait_for_package': False} 2023-12-14T18:14:52.972 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:14:52.973 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'tag': None, 'wait_for_package': False} 2023-12-14T18:14:52.973 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:14:53.115 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/ 2023-12-14T18:14:53.115 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g1faa3533 2023-12-14T18:14:53.186 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/ 2023-12-14T18:14:53.186 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g1faa3533 2023-12-14T18:14:53.189 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/ 2023-12-14T18:14:53.189 INFO:teuthology.task.install.rpm:Package version is 18.0.0-3905.g1faa3533 2023-12-14T18:14:53.221 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-14T18:14:53.222 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:14:53.222 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-14T18:14:53.249 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-14T18:14:53.249 DEBUG:teuthology.orchestra.run.smithi097:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1faa353368ad4f5aeb56d7107b638336f58659ed/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-14T18:14:53.293 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-14T18:14:53.294 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:14:53.294 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-14T18:14:53.316 DEBUG:teuthology.orchestra.run.smithi097:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-14T18:14:53.327 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-14T18:14:53.327 DEBUG:teuthology.orchestra.run.smithi093:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1faa353368ad4f5aeb56d7107b638336f58659ed/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-14T18:14:53.358 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-12-14T18:14:53.358 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:14:53.358 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-12-14T18:14:53.391 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2023-12-14T18:14:53.392 DEBUG:teuthology.orchestra.run.smithi160:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1faa353368ad4f5aeb56d7107b638336f58659ed/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-12-14T18:14:53.395 DEBUG:teuthology.orchestra.run.smithi097:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-14T18:14:53.398 DEBUG:teuthology.orchestra.run.smithi093:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-14T18:14:53.461 INFO:teuthology.orchestra.run.smithi097.stdout:check_obsoletes = 1 2023-12-14T18:14:53.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum clean all 2023-12-14T18:14:53.464 DEBUG:teuthology.orchestra.run.smithi160:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-12-14T18:14:53.479 DEBUG:teuthology.orchestra.run.smithi093:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-14T18:14:53.547 INFO:teuthology.orchestra.run.smithi093.stdout:check_obsoletes = 1 2023-12-14T18:14:53.549 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum clean all 2023-12-14T18:14:53.550 DEBUG:teuthology.orchestra.run.smithi160:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-12-14T18:14:53.620 INFO:teuthology.orchestra.run.smithi160.stdout:check_obsoletes = 1 2023-12-14T18:14:53.621 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum clean all 2023-12-14T18:14:53.837 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:53.927 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:54.006 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:55.221 INFO:teuthology.orchestra.run.smithi093.stdout:52 files removed 2023-12-14T18:14:55.281 DEBUG:teuthology.orchestra.run.smithi093:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-14T18:14:55.295 INFO:teuthology.orchestra.run.smithi097.stdout:52 files removed 2023-12-14T18:14:55.354 DEBUG:teuthology.orchestra.run.smithi097:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-14T18:14:55.544 INFO:teuthology.orchestra.run.smithi160.stdout:52 files removed 2023-12-14T18:14:55.602 DEBUG:teuthology.orchestra.run.smithi160:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2023-12-14T18:14:55.619 INFO:teuthology.orchestra.run.smithi093.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:55.691 INFO:teuthology.orchestra.run.smithi097.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:55.939 INFO:teuthology.orchestra.run.smithi160.stdout:Updating Subscription Management repositories. 2023-12-14T18:14:57.657 INFO:teuthology.orchestra.run.smithi097.stdout:ceph packages for x86_64 209 kB/s | 80 kB 00:00 2023-12-14T18:14:57.748 INFO:teuthology.orchestra.run.smithi093.stdout:ceph packages for x86_64 243 kB/s | 80 kB 00:00 2023-12-14T18:14:57.815 INFO:teuthology.orchestra.run.smithi160.stdout:ceph packages for x86_64 206 kB/s | 80 kB 00:00 2023-12-14T18:14:57.991 INFO:teuthology.orchestra.run.smithi097.stdout:ceph noarch packages 31 kB/s | 9.4 kB 00:00 2023-12-14T18:14:58.082 INFO:teuthology.orchestra.run.smithi093.stdout:ceph noarch packages 31 kB/s | 9.4 kB 00:00 2023-12-14T18:14:58.116 INFO:teuthology.orchestra.run.smithi160.stdout:ceph noarch packages 35 kB/s | 9.4 kB 00:00 2023-12-14T18:14:58.359 INFO:teuthology.orchestra.run.smithi097.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-12-14T18:14:58.391 INFO:teuthology.orchestra.run.smithi093.stdout:ceph source packages 5.3 kB/s | 1.5 kB 00:00 2023-12-14T18:14:58.483 INFO:teuthology.orchestra.run.smithi160.stdout:ceph source packages 4.4 kB/s | 1.5 kB 00:00 2023-12-14T18:14:58.501 INFO:teuthology.orchestra.run.smithi097.stdout:Copr repo for python3-asyncssh owned by ceph 30 kB/s | 3.5 kB 00:00 2023-12-14T18:14:58.533 INFO:teuthology.orchestra.run.smithi093.stdout:Copr repo for python3-asyncssh owned by ceph 29 kB/s | 3.5 kB 00:00 2023-12-14T18:14:58.634 INFO:teuthology.orchestra.run.smithi160.stdout:Copr repo for python3-asyncssh owned by ceph 28 kB/s | 3.5 kB 00:00 2023-12-14T18:14:59.634 INFO:teuthology.orchestra.run.smithi160.stdout:Extra Packages for Enterprise Linux 16 MB/s | 16 MB 00:00 2023-12-14T18:15:03.293 INFO:teuthology.orchestra.run.smithi160.stdout:lab-extras 236 kB/s | 24 kB 00:00 2023-12-14T18:15:04.054 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2023-12-14T18:15:06.801 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 58 MB 00:01 2023-12-14T18:15:16.285 INFO:teuthology.orchestra.run.smithi093.stdout:Extra Packages for Enterprise Linux 922 kB/s | 16 MB 00:17 2023-12-14T18:15:16.588 INFO:teuthology.orchestra.run.smithi097.stdout:Extra Packages for Enterprise Linux 905 kB/s | 16 MB 00:18 2023-12-14T18:15:17.821 INFO:teuthology.orchestra.run.smithi160.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 64 MB 00:01 2023-12-14T18:15:19.791 INFO:teuthology.orchestra.run.smithi093.stdout:lab-extras 436 kB/s | 24 kB 00:00 2023-12-14T18:15:20.106 INFO:teuthology.orchestra.run.smithi097.stdout:lab-extras 289 kB/s | 24 kB 00:00 2023-12-14T18:15:21.143 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 50 MB/s | 64 MB 00:01 2023-12-14T18:15:21.437 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 MB/s | 58 MB 00:01 2023-12-14T18:15:31.502 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 47 MB/s | 58 MB 00:01 2023-12-14T18:15:31.604 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2023-12-14T18:15:32.276 INFO:teuthology.orchestra.run.smithi160.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:15:32.277 INFO:teuthology.orchestra.run.smithi160.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:15:32.295 INFO:teuthology.orchestra.run.smithi160.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-14T18:15:32.297 INFO:teuthology.orchestra.run.smithi160.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-14T18:15:32.298 INFO:teuthology.orchestra.run.smithi160.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-14T18:15:32.300 INFO:teuthology.orchestra.run.smithi160.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-14T18:15:32.303 INFO:teuthology.orchestra.run.smithi160.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-14T18:15:32.306 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-14T18:15:32.308 INFO:teuthology.orchestra.run.smithi160.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-14T18:15:32.517 INFO:teuthology.orchestra.run.smithi160.stdout:Dependencies resolved. 2023-12-14T18:15:32.531 INFO:teuthology.orchestra.run.smithi160.stdout:=========================================================================================================================== 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout: Package Arch Version Repository Size 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout:=========================================================================================================================== 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout:Installing: 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout: ceph x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.4 k 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 5.3 M 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 847 k 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 165 k 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 1.5 M 2023-12-14T18:15:32.532 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 127 k 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 1.7 M 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 7.4 M 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 47 k 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 13 M 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 41 M 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 265 k 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 83 k 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 258 k 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 18 k 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-14T18:15:32.533 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 31 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 771 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 139 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 811 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 197 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 370 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 364 k 2023-12-14T18:15:32.534 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 107 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 94 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.4 M 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 179 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrading: 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: librados2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.7 M 2023-12-14T18:15:32.535 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.6 M 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: openssl x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 710 k 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout:Installing dependencies: 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 22 M 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 23 k 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 2.3 M 2023-12-14T18:15:32.536 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 240 k 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 4.6 M 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 18 M 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 15 k 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 24 k 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-14T18:15:32.537 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 137 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 473 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.5 M 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-14T18:15:32.538 INFO:teuthology.orchestra.run.smithi160.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-14T18:15:32.539 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 45 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 119 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-14T18:15:32.540 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-14T18:15:32.541 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-14T18:15:32.542 INFO:teuthology.orchestra.run.smithi160.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout:Enabling module streams: 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql 10 2023-12-14T18:15:32.543 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:15:32.544 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction Summary 2023-12-14T18:15:32.544 INFO:teuthology.orchestra.run.smithi160.stdout:=========================================================================================================================== 2023-12-14T18:15:32.544 INFO:teuthology.orchestra.run.smithi160.stdout:Install 129 Packages 2023-12-14T18:15:32.544 INFO:teuthology.orchestra.run.smithi160.stdout:Upgrade 16 Packages 2023-12-14T18:15:32.544 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:15:32.544 INFO:teuthology.orchestra.run.smithi160.stdout:Total download size: 205 M 2023-12-14T18:15:32.544 INFO:teuthology.orchestra.run.smithi160.stdout:Downloading Packages: 2023-12-14T18:15:32.713 INFO:teuthology.orchestra.run.smithi160.stdout:(1/145): ceph-18.0.0-3905.g1faa3533.el8.x86_64. 48 kB/s | 6.4 kB 00:00 2023-12-14T18:15:33.005 INFO:teuthology.orchestra.run.smithi160.stdout:(2/145): ceph-fuse-18.0.0-3905.g1faa3533.el8.x8 2.8 MB/s | 847 kB 00:00 2023-12-14T18:15:33.072 INFO:teuthology.orchestra.run.smithi160.stdout:(3/145): ceph-immutable-object-cache-18.0.0-390 2.4 MB/s | 165 kB 00:00 2023-12-14T18:15:33.414 INFO:teuthology.orchestra.run.smithi160.stdout:(4/145): ceph-mds-18.0.0-3905.g1faa3533.el8.x86 6.7 MB/s | 2.3 MB 00:00 2023-12-14T18:15:33.697 INFO:teuthology.orchestra.run.smithi160.stdout:(5/145): ceph-mgr-18.0.0-3905.g1faa3533.el8.x86 5.5 MB/s | 1.5 MB 00:00 2023-12-14T18:15:34.547 INFO:teuthology.orchestra.run.smithi097.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 48 MB/s | 64 MB 00:01 2023-12-14T18:15:34.632 INFO:teuthology.orchestra.run.smithi160.stdout:(6/145): ceph-base-18.0.0-3905.g1faa3533.el8.x8 2.6 MB/s | 5.3 MB 00:02 2023-12-14T18:15:34.731 INFO:teuthology.orchestra.run.smithi160.stdout:(7/145): ceph-mon-18.0.0-3905.g1faa3533.el8.x86 4.5 MB/s | 4.6 MB 00:01 2023-12-14T18:15:36.265 INFO:teuthology.orchestra.run.smithi160.stdout:(8/145): ceph-common-18.0.0-3905.g1faa3533.el8. 5.9 MB/s | 22 MB 00:03 2023-12-14T18:15:36.365 INFO:teuthology.orchestra.run.smithi160.stdout:(9/145): ceph-selinux-18.0.0-3905.g1faa3533.el8 244 kB/s | 24 kB 00:00 2023-12-14T18:15:36.507 INFO:teuthology.orchestra.run.smithi160.stdout:(10/145): ceph-radosgw-18.0.0-3905.g1faa3533.el 7.3 MB/s | 13 MB 00:01 2023-12-14T18:15:36.599 INFO:teuthology.orchestra.run.smithi160.stdout:(11/145): cephfs-mirror-18.0.0-3905.g1faa3533.e 2.8 MB/s | 258 kB 00:00 2023-12-14T18:15:36.675 INFO:teuthology.orchestra.run.smithi160.stdout:(12/145): libcephfs-devel-18.0.0-3905.g1faa3533 414 kB/s | 31 kB 00:00 2023-12-14T18:15:36.824 INFO:teuthology.orchestra.run.smithi160.stdout:(13/145): libcephfs2-18.0.0-3905.g1faa3533.el8. 5.0 MB/s | 771 kB 00:00 2023-12-14T18:15:36.921 INFO:teuthology.orchestra.run.smithi160.stdout:(14/145): libcephsqlite-18.0.0-3905.g1faa3533.e 1.4 MB/s | 137 kB 00:00 2023-12-14T18:15:37.001 INFO:teuthology.orchestra.run.smithi160.stdout:(15/145): librados-devel-18.0.0-3905.g1faa3533. 1.7 MB/s | 139 kB 00:00 2023-12-14T18:15:37.142 INFO:teuthology.orchestra.run.smithi160.stdout:(16/145): libradosstriper1-18.0.0-3905.g1faa353 3.3 MB/s | 473 kB 00:00 2023-12-14T18:15:38.335 INFO:teuthology.orchestra.run.smithi160.stdout:(17/145): librgw2-18.0.0-3905.g1faa3533.el8.x86 5.5 MB/s | 6.5 MB 00:01 2023-12-14T18:15:38.417 INFO:teuthology.orchestra.run.smithi160.stdout:(18/145): python3-ceph-argparse-18.0.0-3905.g1f 547 kB/s | 45 kB 00:00 2023-12-14T18:15:38.793 INFO:teuthology.orchestra.run.smithi160.stdout:(19/145): ceph-test-18.0.0-3905.g1faa3533.el8.x 17 MB/s | 41 MB 00:02 2023-12-14T18:15:38.818 INFO:teuthology.orchestra.run.smithi160.stdout:(20/145): python3-ceph-common-18.0.0-3905.g1faa 296 kB/s | 119 kB 00:00 2023-12-14T18:15:38.877 INFO:teuthology.orchestra.run.smithi160.stdout:(21/145): python3-cephfs-18.0.0-3905.g1faa3533. 2.3 MB/s | 197 kB 00:00 2023-12-14T18:15:38.902 INFO:teuthology.orchestra.run.smithi160.stdout:(22/145): python3-rados-18.0.0-3905.g1faa3533.e 4.3 MB/s | 370 kB 00:00 2023-12-14T18:15:38.952 INFO:teuthology.orchestra.run.smithi160.stdout:(23/145): python3-rgw-18.0.0-3905.g1faa3533.el8 2.1 MB/s | 107 kB 00:00 2023-12-14T18:15:38.977 INFO:teuthology.orchestra.run.smithi160.stdout:(24/145): python3-rbd-18.0.0-3905.g1faa3533.el8 3.5 MB/s | 364 kB 00:00 2023-12-14T18:15:39.011 INFO:teuthology.orchestra.run.smithi160.stdout:(25/145): rbd-fuse-18.0.0-3905.g1faa3533.el8.x8 1.6 MB/s | 94 kB 00:00 2023-12-14T18:15:39.086 INFO:teuthology.orchestra.run.smithi160.stdout:(26/145): rbd-nbd-18.0.0-3905.g1faa3533.el8.x86 2.3 MB/s | 179 kB 00:00 2023-12-14T18:15:39.178 INFO:teuthology.orchestra.run.smithi160.stdout:(27/145): ceph-grafana-dashboards-18.0.0-3905.g 252 kB/s | 23 kB 00:00 2023-12-14T18:15:39.245 INFO:teuthology.orchestra.run.smithi160.stdout:(28/145): ceph-mgr-cephadm-18.0.0-3905.g1faa353 1.9 MB/s | 127 kB 00:00 2023-12-14T18:15:39.362 INFO:teuthology.orchestra.run.smithi160.stdout:(29/145): rbd-mirror-18.0.0-3905.g1faa3533.el8. 9.0 MB/s | 3.4 MB 00:00 2023-12-14T18:15:39.521 INFO:teuthology.orchestra.run.smithi160.stdout:(30/145): ceph-mgr-dashboard-18.0.0-3905.g1faa3 6.2 MB/s | 1.7 MB 00:00 2023-12-14T18:15:39.587 INFO:teuthology.orchestra.run.smithi160.stdout:(31/145): ceph-mgr-modules-core-18.0.0-3905.g1f 3.5 MB/s | 240 kB 00:00 2023-12-14T18:15:39.663 INFO:teuthology.orchestra.run.smithi160.stdout:(32/145): ceph-mgr-rook-18.0.0-3905.g1faa3533.e 632 kB/s | 47 kB 00:00 2023-12-14T18:15:39.721 INFO:teuthology.orchestra.run.smithi160.stdout:(33/145): ceph-prometheus-alerts-18.0.0-3905.g1 250 kB/s | 15 kB 00:00 2023-12-14T18:15:39.780 INFO:teuthology.orchestra.run.smithi160.stdout:(34/145): ceph-volume-18.0.0-3905.g1faa3533.el8 4.5 MB/s | 265 kB 00:00 2023-12-14T18:15:39.908 INFO:teuthology.orchestra.run.smithi160.stdout:(35/145): ceph-mgr-diskprediction-local-18.0.0- 14 MB/s | 7.4 MB 00:00 2023-12-14T18:15:40.073 INFO:teuthology.orchestra.run.smithi160.stdout:(36/145): cephadm-18.0.0-3905.g1faa3533.el8.noa 286 kB/s | 83 kB 00:00 2023-12-14T18:15:40.098 INFO:teuthology.orchestra.run.smithi160.stdout:(37/145): cephfs-top-18.0.0-3905.g1faa3533.el8. 95 kB/s | 18 kB 00:00 2023-12-14T18:15:40.256 INFO:teuthology.orchestra.run.smithi160.stdout:(38/145): liboath-2.6.2-3.el8.x86_64.rpm 372 kB/s | 59 kB 00:00 2023-12-14T18:15:40.307 INFO:teuthology.orchestra.run.smithi160.stdout:(39/145): gperftools-libs-2.7-9.el8.x86_64.rpm 1.3 MB/s | 306 kB 00:00 2023-12-14T18:15:40.332 INFO:teuthology.orchestra.run.smithi160.stdout:(40/145): libunwind-1.3.1-3.el8.x86_64.rpm 1.0 MB/s | 75 kB 00:00 2023-12-14T18:15:40.391 INFO:teuthology.orchestra.run.smithi160.stdout:(41/145): python3-asyncssh-2.7.0-2.el8.noarch.r 5.6 MB/s | 476 kB 00:00 2023-12-14T18:15:40.416 INFO:teuthology.orchestra.run.smithi160.stdout:(42/145): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 522 kB/s | 44 kB 00:00 2023-12-14T18:15:40.441 INFO:teuthology.orchestra.run.smithi160.stdout:(43/145): python3-beautifulsoup4-4.6.3-2.el8.1. 3.6 MB/s | 185 kB 00:00 2023-12-14T18:15:40.500 INFO:teuthology.orchestra.run.smithi160.stdout:(44/145): python3-cachetools-3.1.1-4.el8.noarch 395 kB/s | 33 kB 00:00 2023-12-14T18:15:40.525 INFO:teuthology.orchestra.run.smithi160.stdout:(45/145): python3-certifi-2018.10.15-7.el8.noar 195 kB/s | 16 kB 00:00 2023-12-14T18:15:40.575 INFO:teuthology.orchestra.run.smithi160.stdout:(46/145): python3-cheroot-8.5.2-1.el8.noarch.rp 2.2 MB/s | 173 kB 00:00 2023-12-14T18:15:40.601 INFO:teuthology.orchestra.run.smithi160.stdout:(47/145): python3-cherrypy-18.4.0-1.el8.noarch. 5.0 MB/s | 384 kB 00:00 2023-12-14T18:15:40.626 INFO:teuthology.orchestra.run.smithi160.stdout:(48/145): python3-google-auth-1.1.1-10.el8.noar 1.6 MB/s | 81 kB 00:00 2023-12-14T18:15:40.643 INFO:teuthology.orchestra.run.smithi160.stdout:(49/145): python3-jaraco-6.2-6.el8.noarch.rpm 267 kB/s | 11 kB 00:00 2023-12-14T18:15:40.668 INFO:teuthology.orchestra.run.smithi160.stdout:(50/145): python3-jaraco-functools-2.0-4.el8.no 422 kB/s | 18 kB 00:00 2023-12-14T18:15:40.752 INFO:teuthology.orchestra.run.smithi160.stdout:(51/145): python3-kubernetes-11.0.0-6.el8.noarc 11 MB/s | 1.2 MB 00:00 2023-12-14T18:15:40.777 INFO:teuthology.orchestra.run.smithi160.stdout:(52/145): python3-logutils-0.3.5-11.el8.noarch. 449 kB/s | 49 kB 00:00 2023-12-14T18:15:40.794 INFO:teuthology.orchestra.run.smithi160.stdout:(53/145): python3-more-itertools-7.2.0-3.el8.no 1.4 MB/s | 59 kB 00:00 2023-12-14T18:15:40.819 INFO:teuthology.orchestra.run.smithi160.stdout:(54/145): python3-natsort-7.1.1-2.el8.noarch.rp 1.4 MB/s | 60 kB 00:00 2023-12-14T18:15:40.844 INFO:teuthology.orchestra.run.smithi160.stdout:(55/145): python3-pecan-1.3.2-9.el8.noarch.rpm 5.5 MB/s | 283 kB 00:00 2023-12-14T18:15:40.861 INFO:teuthology.orchestra.run.smithi160.stdout:(56/145): python3-portend-2.6-1.el8.noarch.rpm 372 kB/s | 16 kB 00:00 2023-12-14T18:15:40.887 INFO:teuthology.orchestra.run.smithi160.stdout:(57/145): python3-repoze-lru-0.7-6.el8.noarch.r 802 kB/s | 34 kB 00:00 2023-12-14T18:15:41.087 INFO:teuthology.orchestra.run.smithi160.stdout:(58/145): ceph-osd-18.0.0-3905.g1faa3533.el8.x8 2.7 MB/s | 18 MB 00:06 2023-12-14T18:15:41.112 INFO:teuthology.orchestra.run.smithi160.stdout:(59/145): python3-rsa-4.9-2.el8.noarch.rpm 285 kB/s | 64 kB 00:00 2023-12-14T18:15:41.146 INFO:teuthology.orchestra.run.smithi160.stdout:(60/145): python3-routes-2.4.1-12.el8.noarch.rp 691 kB/s | 196 kB 00:00 2023-12-14T18:15:41.171 INFO:teuthology.orchestra.run.smithi160.stdout:(61/145): python3-singledispatch-3.4.0.3-18.el8 443 kB/s | 24 kB 00:00 2023-12-14T18:15:41.196 INFO:teuthology.orchestra.run.smithi160.stdout:(62/145): python3-tempora-1.14.1-5.el8.noarch.r 591 kB/s | 29 kB 00:00 2023-12-14T18:15:41.222 INFO:teuthology.orchestra.run.smithi160.stdout:(63/145): python3-simplegeneric-0.8.1-17.el8.no 140 kB/s | 19 kB 00:00 2023-12-14T18:15:41.247 INFO:teuthology.orchestra.run.smithi160.stdout:(64/145): python3-trustme-0.6.0-4.el8.noarch.rp 366 kB/s | 27 kB 00:00 2023-12-14T18:15:41.272 INFO:teuthology.orchestra.run.smithi160.stdout:(65/145): python3-waitress-1.4.3-1.el8.noarch.r 3.3 MB/s | 255 kB 00:00 2023-12-14T18:15:41.298 INFO:teuthology.orchestra.run.smithi160.stdout:(66/145): python3-websocket-client-0.56.0-5.el8 1.2 MB/s | 61 kB 00:00 2023-12-14T18:15:41.342 INFO:teuthology.orchestra.run.smithi160.stdout:(67/145): python3-webtest-2.0.33-1.el8.noarch.r 1.2 MB/s | 85 kB 00:00 2023-12-14T18:15:41.373 INFO:teuthology.orchestra.run.smithi160.stdout:(68/145): python3-zc-lockfile-2.0-2.el8.noarch. 305 kB/s | 23 kB 00:00 2023-12-14T18:15:41.398 INFO:teuthology.orchestra.run.smithi160.stdout:(69/145): python3-webob-1.8.5-1.el8.1.noarch.rp 1.4 MB/s | 251 kB 00:00 2023-12-14T18:15:41.424 INFO:teuthology.orchestra.run.smithi160.stdout:(70/145): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-12-14T18:15:41.482 INFO:teuthology.orchestra.run.smithi160.stdout:(71/145): thrift-0.13.0-2.el8.x86_64.rpm 12 MB/s | 1.7 MB 00:00 2023-12-14T18:15:41.541 INFO:teuthology.orchestra.run.smithi160.stdout:(72/145): python3-jsonpointer-1.10-11.el8.noarc 141 kB/s | 20 kB 00:00 2023-12-14T18:15:41.649 INFO:teuthology.orchestra.run.smithi160.stdout:(73/145): python3-prettytable-0.7.2-14.el8.noar 200 kB/s | 44 kB 00:00 2023-12-14T18:15:41.675 INFO:teuthology.orchestra.run.smithi160.stdout:(74/145): python3-jsonpatch-1.21-2.el8.noarch.r 199 kB/s | 27 kB 00:00 2023-12-14T18:15:41.700 INFO:teuthology.orchestra.run.smithi160.stdout:(75/145): python3-pyasn1-modules-0.3.7-6.el8.no 519 kB/s | 110 kB 00:00 2023-12-14T18:15:41.718 INFO:teuthology.orchestra.run.smithi093.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 16 MB/s | 8.8 MB 00:00 2023-12-14T18:15:41.784 INFO:teuthology.orchestra.run.smithi160.stdout:(76/145): python3-pyasn1-0.3.7-6.el8.noarch.rpm 944 kB/s | 126 kB 00:00 2023-12-14T18:15:41.817 INFO:teuthology.orchestra.run.smithi160.stdout:(77/145): bison-3.0.4-10.el8.x86_64.rpm 5.8 MB/s | 688 kB 00:00 2023-12-14T18:15:41.851 INFO:teuthology.orchestra.run.smithi160.stdout:(78/145): python3-werkzeug-0.12.2-4.el8.noarch. 2.5 MB/s | 457 kB 00:00 2023-12-14T18:15:41.926 INFO:teuthology.orchestra.run.smithi160.stdout:(79/145): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2023-12-14T18:15:41.968 INFO:teuthology.orchestra.run.smithi160.stdout:(80/145): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2023-12-14T18:15:41.993 INFO:teuthology.orchestra.run.smithi160.stdout:(81/145): python3-markupsafe-0.23-19.el8.x86_64 273 kB/s | 39 kB 00:00 2023-12-14T18:15:42.060 INFO:teuthology.orchestra.run.smithi160.stdout:(82/145): oniguruma-6.8.2-2.el8.x86_64.rpm 2.0 MB/s | 187 kB 00:00 2023-12-14T18:15:42.085 INFO:teuthology.orchestra.run.smithi160.stdout:(83/145): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.6 MB/s | 259 kB 00:00 2023-12-14T18:15:42.136 INFO:teuthology.orchestra.run.smithi160.stdout:(84/145): uuid-1.6.2-43.el8.x86_64.rpm 450 kB/s | 64 kB 00:00 2023-12-14T18:15:42.344 INFO:teuthology.orchestra.run.smithi160.stdout:(85/145): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-12-14T18:15:42.361 INFO:teuthology.orchestra.run.smithi160.stdout:(86/145): python3-pyOpenSSL-19.0.0-1.el8.noarch 344 kB/s | 103 kB 00:00 2023-12-14T18:15:42.503 INFO:teuthology.orchestra.run.smithi160.stdout:(87/145): python3-jinja2-2.10.1-3.el8.noarch.rp 3.3 MB/s | 538 kB 00:00 2023-12-14T18:15:42.687 INFO:teuthology.orchestra.run.smithi160.stdout:(88/145): python3-scipy-1.0.0-21.module+el8.5.0 23 MB/s | 14 MB 00:00 2023-12-14T18:15:42.779 INFO:teuthology.orchestra.run.smithi160.stdout:(89/145): python3-numpy-1.14.3-10.el8.x86_64.rp 9.0 MB/s | 3.7 MB 00:00 2023-12-14T18:15:42.821 INFO:teuthology.orchestra.run.smithi160.stdout:(90/145): socat-1.7.4.1-1.el8.x86_64.rpm 1.0 MB/s | 323 kB 00:00 2023-12-14T18:15:42.904 INFO:teuthology.orchestra.run.smithi160.stdout:(91/145): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2023-12-14T18:15:42.921 INFO:teuthology.orchestra.run.smithi160.stdout:(92/145): python3-numpy-f2py-1.14.3-10.el8.x86_ 962 kB/s | 225 kB 00:00 2023-12-14T18:15:43.047 INFO:teuthology.orchestra.run.smithi160.stdout:(93/145): python3-mako-1.0.6-14.el8.noarch.rpm 1.1 MB/s | 157 kB 00:00 2023-12-14T18:15:43.132 INFO:teuthology.orchestra.run.smithi160.stdout:(94/145): openblas-threads-0.3.15-6.el8.x86_64. 17 MB/s | 5.2 MB 00:00 2023-12-14T18:15:43.230 INFO:teuthology.orchestra.run.smithi160.stdout:(95/145): openblas-0.3.15-6.el8.x86_64.rpm 16 MB/s | 5.0 MB 00:00 2023-12-14T18:15:43.272 INFO:teuthology.orchestra.run.smithi160.stdout:(96/145): postgresql-10.23-2.module+el8.8.0+194 6.7 MB/s | 1.5 MB 00:00 2023-12-14T18:15:43.406 INFO:teuthology.orchestra.run.smithi160.stdout:(97/145): postgresql-contrib-10.23-2.module+el8 2.9 MB/s | 811 kB 00:00 2023-12-14T18:15:43.489 INFO:teuthology.orchestra.run.smithi160.stdout:(98/145): postgresql-server-10.23-2.module+el8. 20 MB/s | 5.1 MB 00:00 2023-12-14T18:15:43.523 INFO:teuthology.orchestra.run.smithi160.stdout:(99/145): automake-1.16.1-8.el8.noarch.rpm 2.8 MB/s | 714 kB 00:00 2023-12-14T18:15:43.548 INFO:teuthology.orchestra.run.smithi160.stdout:(100/145): libpq-13.11-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2023-12-14T18:15:43.640 INFO:teuthology.orchestra.run.smithi160.stdout:(101/145): python3-pytz-2017.2-11.el8.noarch.rp 464 kB/s | 54 kB 00:00 2023-12-14T18:15:43.665 INFO:teuthology.orchestra.run.smithi160.stdout:(102/145): jq-1.6-7.el8.x86_64.rpm 1.1 MB/s | 202 kB 00:00 2023-12-14T18:15:43.707 INFO:teuthology.orchestra.run.smithi160.stdout:(103/145): libverto-devel-0.3.0-5.el8.x86_64.rp 113 kB/s | 18 kB 00:00 2023-12-14T18:15:43.791 INFO:teuthology.orchestra.run.smithi160.stdout:(104/145): python3-jwt-1.6.1-2.el8.noarch.rpm 283 kB/s | 43 kB 00:00 2023-12-14T18:15:43.883 INFO:teuthology.orchestra.run.smithi160.stdout:(105/145): python3-pycparser-2.14-14.el8.noarch 622 kB/s | 109 kB 00:00 2023-12-14T18:15:43.900 INFO:teuthology.orchestra.run.smithi160.stdout:(106/145): python3-oauthlib-2.1.0-1.el8.noarch. 661 kB/s | 155 kB 00:00 2023-12-14T18:15:43.925 INFO:teuthology.orchestra.run.smithi160.stdout:(107/145): python3-requests-oauthlib-1.0.0-1.el 319 kB/s | 43 kB 00:00 2023-12-14T18:15:44.001 INFO:teuthology.orchestra.run.smithi160.stdout:(108/145): libaio-devel-0.3.112-1.el8.x86_64.rp 159 kB/s | 19 kB 00:00 2023-12-14T18:15:44.051 INFO:teuthology.orchestra.run.smithi160.stdout:(109/145): libzstd-devel-1.4.4-1.el8.x86_64.rpm 291 kB/s | 44 kB 00:00 2023-12-14T18:15:44.092 INFO:teuthology.orchestra.run.smithi160.stdout:(110/145): pcre2-devel-10.32-2.el8.x86_64.rpm 3.5 MB/s | 605 kB 00:00 2023-12-14T18:15:44.148 INFO:teuthology.orchestra.run.smithi160.stdout:(111/145): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.5 MB/s | 229 kB 00:00 2023-12-14T18:15:44.201 INFO:teuthology.orchestra.run.smithi160.stdout:(112/145): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2023-12-14T18:15:44.260 INFO:teuthology.orchestra.run.smithi160.stdout:(113/145): libselinux-devel-2.9-5.el8.x86_64.rp 1.2 MB/s | 200 kB 00:00 2023-12-14T18:15:44.294 INFO:teuthology.orchestra.run.smithi160.stdout:(114/145): keyutils-libs-devel-1.5.10-9.el8.x86 331 kB/s | 48 kB 00:00 2023-12-14T18:15:44.353 INFO:teuthology.orchestra.run.smithi160.stdout:(115/145): libsepol-devel-2.9-3.el8.x86_64.rpm 578 kB/s | 87 kB 00:00 2023-12-14T18:15:44.377 INFO:teuthology.orchestra.run.smithi160.stdout:(116/145): zlib-devel-1.2.11-18.el8_5.x86_64.rp 492 kB/s | 58 kB 00:00 2023-12-14T18:15:44.486 INFO:teuthology.orchestra.run.smithi160.stdout:(117/145): libcom_err-devel-1.45.6-4.el8.x86_64 201 kB/s | 39 kB 00:00 2023-12-14T18:15:44.511 INFO:teuthology.orchestra.run.smithi160.stdout:(118/145): fuse-2.9.7-17.el8.x86_64.rpm 622 kB/s | 83 kB 00:00 2023-12-14T18:15:44.553 INFO:teuthology.orchestra.run.smithi160.stdout:(119/145): openssl-devel-1.1.1k-9.el8_7.x86_64. 12 MB/s | 2.3 MB 00:00 2023-12-14T18:15:44.579 INFO:teuthology.orchestra.run.smithi160.stdout:(120/145): libquadmath-8.5.0-20.el8.x86_64.rpm 2.5 MB/s | 172 kB 00:00 2023-12-14T18:15:44.620 INFO:teuthology.orchestra.run.smithi160.stdout:(121/145): libgfortran-8.5.0-20.el8.x86_64.rpm 4.7 MB/s | 645 kB 00:00 2023-12-14T18:15:44.646 INFO:teuthology.orchestra.run.smithi160.stdout:(122/145): libuuid-devel-2.32.1-43.el8.x86_64.r 1.1 MB/s | 100 kB 00:00 2023-12-14T18:15:44.754 INFO:teuthology.orchestra.run.smithi160.stdout:(123/145): xfsprogs-devel-5.0.0-12.el8.x86_64.r 670 kB/s | 73 kB 00:00 2023-12-14T18:15:44.780 INFO:teuthology.orchestra.run.smithi160.stdout:(124/145): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.2 MB/s | 238 kB 00:00 2023-12-14T18:15:44.913 INFO:teuthology.orchestra.run.smithi160.stdout:(125/145): elfutils-libelf-devel-0.189-3.el8.x8 463 kB/s | 62 kB 00:00 2023-12-14T18:15:44.947 INFO:teuthology.orchestra.run.smithi160.stdout:(126/145): python3-cryptography-3.2.1-6.el8.x86 1.7 MB/s | 559 kB 00:00 2023-12-14T18:15:44.992 INFO:teuthology.orchestra.run.smithi160.stdout:(127/145): NetworkManager-initscripts-updown-1. 604 kB/s | 143 kB 00:00 2023-12-14T18:15:45.381 INFO:teuthology.orchestra.run.smithi160.stdout:(128/145): librabbitmq-0.9.0-4.el8.x86_64.rpm 100 kB/s | 47 kB 00:00 2023-12-14T18:15:45.414 INFO:teuthology.orchestra.run.smithi160.stdout:(129/145): krb5-devel-1.18.2-26.el8_9.x86_64.rp 1.2 MB/s | 562 kB 00:00 2023-12-14T18:15:45.573 INFO:teuthology.orchestra.run.smithi160.stdout:(130/145): librados2-18.0.0-3905.g1faa3533.el8. 6.3 MB/s | 3.7 MB 00:00 2023-12-14T18:15:45.615 INFO:teuthology.orchestra.run.smithi160.stdout:(131/145): openssl-1.1.1k-9.el8_7.x86_64.rpm 3.5 MB/s | 710 kB 00:00 2023-12-14T18:15:45.757 INFO:teuthology.orchestra.run.smithi160.stdout:(132/145): openssl-libs-1.1.1k-9.el8_7.x86_64.r 8.0 MB/s | 1.5 MB 00:00 2023-12-14T18:15:45.782 INFO:teuthology.orchestra.run.smithi160.stdout:(133/145): elfutils-libelf-0.189-3.el8.x86_64.r 1.4 MB/s | 232 kB 00:00 2023-12-14T18:15:45.841 INFO:teuthology.orchestra.run.smithi160.stdout:(134/145): elfutils-libs-0.189-3.el8.x86_64.rpm 3.6 MB/s | 303 kB 00:00 2023-12-14T18:15:45.874 INFO:teuthology.orchestra.run.smithi160.stdout:(135/145): libmount-2.32.1-43.el8.x86_64.rpm 2.5 MB/s | 237 kB 00:00 2023-12-14T18:15:46.017 INFO:teuthology.orchestra.run.smithi160.stdout:(136/145): libsmartcols-2.32.1-43.el8.x86_64.rp 1.0 MB/s | 179 kB 00:00 2023-12-14T18:15:46.042 INFO:teuthology.orchestra.run.smithi160.stdout:(137/145): libuuid-2.32.1-43.el8.x86_64.rpm 593 kB/s | 99 kB 00:00 2023-12-14T18:15:46.175 INFO:teuthology.orchestra.run.smithi160.stdout:(138/145): util-linux-2.32.1-43.el8.x86_64.rpm 16 MB/s | 2.5 MB 00:00 2023-12-14T18:15:46.226 INFO:teuthology.orchestra.run.smithi160.stdout:(139/145): util-linux-user-2.32.1-43.el8.x86_64 559 kB/s | 103 kB 00:00 2023-12-14T18:15:46.326 INFO:teuthology.orchestra.run.smithi160.stdout:(140/145): librbd1-18.0.0-3905.g1faa3533.el8.x8 3.8 MB/s | 3.6 MB 00:00 2023-12-14T18:15:46.359 INFO:teuthology.orchestra.run.smithi160.stdout:(141/145): xfsdump-3.1.8-6.el8.x86_64.rpm 1.8 MB/s | 333 kB 00:00 2023-12-14T18:15:46.393 INFO:teuthology.orchestra.run.smithi160.stdout:(142/145): iproute-6.2.0-5.el8_9.x86_64.rpm 5.0 MB/s | 854 kB 00:00 2023-12-14T18:15:46.418 INFO:teuthology.orchestra.run.smithi160.stdout:(143/145): libblkid-2.32.1-43.el8.x86_64.rpm 2.3 MB/s | 220 kB 00:00 2023-12-14T18:15:46.518 INFO:teuthology.orchestra.run.smithi160.stdout:(144/145): libfdisk-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 254 kB 00:00 2023-12-14T18:15:46.610 INFO:teuthology.orchestra.run.smithi160.stdout:(145/145): xfsprogs-5.0.0-12.el8.x86_64.rpm 4.9 MB/s | 1.1 MB 00:00 2023-12-14T18:15:46.615 INFO:teuthology.orchestra.run.smithi160.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:15:46.615 INFO:teuthology.orchestra.run.smithi160.stdout:Total 15 MB/s | 205 MB 00:14 2023-12-14T18:15:47.697 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction check 2023-12-14T18:15:48.094 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction check succeeded. 2023-12-14T18:15:48.094 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction test 2023-12-14T18:15:48.524 INFO:teuthology.orchestra.run.smithi093.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:15:48.525 INFO:teuthology.orchestra.run.smithi093.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:15:48.542 INFO:teuthology.orchestra.run.smithi093.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-14T18:15:48.544 INFO:teuthology.orchestra.run.smithi093.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-14T18:15:48.545 INFO:teuthology.orchestra.run.smithi093.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-14T18:15:48.546 INFO:teuthology.orchestra.run.smithi093.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-14T18:15:48.549 INFO:teuthology.orchestra.run.smithi093.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-14T18:15:48.553 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-14T18:15:48.554 INFO:teuthology.orchestra.run.smithi093.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-14T18:15:48.763 INFO:teuthology.orchestra.run.smithi093.stdout:Dependencies resolved. 2023-12-14T18:15:48.776 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-12-14T18:15:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: Package Arch Version Repository Size 2023-12-14T18:15:48.777 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-12-14T18:15:48.777 INFO:teuthology.orchestra.run.smithi093.stdout:Installing: 2023-12-14T18:15:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-14T18:15:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-14T18:15:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: ceph x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.4 k 2023-12-14T18:15:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 5.3 M 2023-12-14T18:15:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 847 k 2023-12-14T18:15:48.777 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 165 k 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 1.5 M 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 127 k 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 1.7 M 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 7.4 M 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 47 k 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 13 M 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 41 M 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 265 k 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 83 k 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 258 k 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 18 k 2023-12-14T18:15:48.778 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 31 k 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 771 k 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 139 k 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 811 k 2023-12-14T18:15:48.779 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 197 k 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 370 k 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 364 k 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 107 k 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 94 k 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.4 M 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 179 k 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrading: 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-14T18:15:48.780 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: librados2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.7 M 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.6 M 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: openssl x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 710 k 2023-12-14T18:15:48.781 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-libs x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout:Installing dependencies: 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 22 M 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 23 k 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 2.3 M 2023-12-14T18:15:48.782 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 240 k 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 4.6 M 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 18 M 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 15 k 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 24 k 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-14T18:15:48.783 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 137 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 473 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.5 M 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-14T18:15:48.784 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:15:48.785 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 45 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 119 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-14T18:15:48.786 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-14T18:15:48.787 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:15:48.788 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-14T18:15:48.789 INFO:teuthology.orchestra.run.smithi093.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling module streams: 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql 10 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction Summary 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout:=========================================================================================================================== 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout:Install 129 Packages 2023-12-14T18:15:48.790 INFO:teuthology.orchestra.run.smithi093.stdout:Upgrade 16 Packages 2023-12-14T18:15:48.791 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:15:48.791 INFO:teuthology.orchestra.run.smithi093.stdout:Total download size: 205 M 2023-12-14T18:15:48.791 INFO:teuthology.orchestra.run.smithi093.stdout:Downloading Packages: 2023-12-14T18:15:48.944 INFO:teuthology.orchestra.run.smithi093.stdout:(1/145): ceph-18.0.0-3905.g1faa3533.el8.x86_64. 55 kB/s | 6.4 kB 00:00 2023-12-14T18:15:48.958 INFO:teuthology.orchestra.run.smithi097.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:15:48.960 INFO:teuthology.orchestra.run.smithi097.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-12-14T18:15:48.977 INFO:teuthology.orchestra.run.smithi097.stdout:Package NetworkManager-1:1.36.0-4.el8.x86_64 is already installed. 2023-12-14T18:15:48.978 INFO:teuthology.orchestra.run.smithi097.stdout:Package iproute-5.15.0-4.el8.x86_64 is already installed. 2023-12-14T18:15:48.979 INFO:teuthology.orchestra.run.smithi097.stdout:Package util-linux-2.32.1-35.el8.x86_64 is already installed. 2023-12-14T18:15:48.981 INFO:teuthology.orchestra.run.smithi097.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-12-14T18:15:48.983 INFO:teuthology.orchestra.run.smithi097.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-12-14T18:15:48.987 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-12-14T18:15:48.988 INFO:teuthology.orchestra.run.smithi097.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-12-14T18:15:49.192 INFO:teuthology.orchestra.run.smithi097.stdout:Dependencies resolved. 2023-12-14T18:15:49.206 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-12-14T18:15:49.206 INFO:teuthology.orchestra.run.smithi097.stdout: Package Arch Version Repository Size 2023-12-14T18:15:49.206 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout:Installing: 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 rhel-8-for-x86_64-baseos-rpms 143 k 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: bison x86_64 3.0.4-10.el8 rhel-8-for-x86_64-appstream-rpms 688 k 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: ceph x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.4 k 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 5.3 M 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 847 k 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 165 k 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 1.5 M 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 127 k 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 1.7 M 2023-12-14T18:15:49.207 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 7.4 M 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 47 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 13 M 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 41 M 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 265 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 83 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 258 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 18 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 62 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: flex x86_64 2.6.1-9.el8 rhel-8-for-x86_64-appstream-rpms 320 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel x86_64 0.3.112-1.el8 rhel-8-for-x86_64-baseos-rpms 19 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 31 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 771 k 2023-12-14T18:15:49.208 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 139 k 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: libtool x86_64 2.4.6-25.el8 rhel-8-for-x86_64-appstream-rpms 709 k 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 100 k 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 2.3 M 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 1.5 M 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 811 k 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server x86_64 10.23-2.module+el8.8.0+19493+5804baf6 rhel-8-for-x86_64-appstream-rpms 5.1 M 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 197 k 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 370 k 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 364 k 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 107 k 2023-12-14T18:15:49.209 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 94 k 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.4 M 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 179 k 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 73 k 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrading: 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 232 k 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs x86_64 0.189-3.el8 rhel-8-for-x86_64-baseos-rpms 303 k 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: iproute x86_64 6.2.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 854 k 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 254 k 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: libmount x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 237 k 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: librados2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.7 M 2023-12-14T18:15:49.210 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 3.6 M 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 179 k 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 99 k 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: openssl x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 710 k 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs x86_64 1:1.1.1k-9.el8_7 rhel-8-for-x86_64-baseos-rpms 1.5 M 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 2.5 M 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user x86_64 2.32.1-43.el8 rhel-8-for-x86_64-baseos-rpms 103 k 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump x86_64 3.1.8-6.el8 rhel-8-for-x86_64-baseos-rpms 333 k 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs x86_64 5.0.0-12.el8 rhel-8-for-x86_64-baseos-rpms 1.1 M 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout:Installing dependencies: 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: automake noarch 1.16.1-8.el8 rhel-8-for-x86_64-appstream-rpms 714 k 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 22 M 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 23 k 2023-12-14T18:15:49.211 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 2.3 M 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 240 k 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 4.6 M 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 18 M 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts noarch 2:18.0.0-3905.g1faa3533.el8 ceph-noarch 15 k 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 24 k 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 rhel-8-for-x86_64-baseos-rpms 48 k 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel x86_64 1.18.2-26.el8_9 rhel-8-for-x86_64-baseos-rpms 562 k 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 137 k 2023-12-14T18:15:49.212 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel x86_64 1.45.6-4.el8 rhel-8-for-x86_64-baseos-rpms 39 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: libpq x86_64 13.11-1.el8 rhel-8-for-x86_64-appstream-rpms 198 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq x86_64 0.9.0-4.el8 rhel-8-for-x86_64-baseos-rpms 47 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 473 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2 x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 6.5 M 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel x86_64 2.9-5.el8 rhel-8-for-x86_64-baseos-rpms 200 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel x86_64 2.9-3.el8 rhel-8-for-x86_64-baseos-rpms 87 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-12-14T18:15:49.213 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel x86_64 0.3.0-5.el8 rhel-8-for-x86_64-baseos-rpms 18 k 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel x86_64 1.4.4-1.el8 rhel-8-for-x86_64-baseos-rpms 44 k 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma x86_64 6.8.2-2.el8 rhel-8-for-x86_64-appstream-rpms 187 k 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 605 k 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 229 k 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32 x86_64 10.32-2.el8 rhel-8-for-x86_64-baseos-rpms 220 k 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-12-14T18:15:49.214 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 45 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common x86_64 2:18.0.0-3905.g1faa3533.el8 ceph 119 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography x86_64 3.2.1-6.el8 rhel-8-for-x86_64-baseos-rpms 559 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2023-12-14T18:15:49.215 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-12-14T18:15:49.216 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy x86_64 1.0.0-21.module+el8.5.0+10916+41bd434d rhel-8-for-x86_64-appstream-rpms 14 M 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-12-14T18:15:49.217 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: uuid x86_64 1.6.2-43.el8 rhel-8-for-x86_64-appstream-rpms 64 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel x86_64 1.2.11-18.el8_5 rhel-8-for-x86_64-baseos-rpms 58 k 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout:Enabling module streams: 2023-12-14T18:15:49.218 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql 10 2023-12-14T18:15:49.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:15:49.219 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction Summary 2023-12-14T18:15:49.219 INFO:teuthology.orchestra.run.smithi097.stdout:=========================================================================================================================== 2023-12-14T18:15:49.219 INFO:teuthology.orchestra.run.smithi097.stdout:Install 129 Packages 2023-12-14T18:15:49.219 INFO:teuthology.orchestra.run.smithi097.stdout:Upgrade 16 Packages 2023-12-14T18:15:49.219 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:15:49.220 INFO:teuthology.orchestra.run.smithi093.stdout:(2/145): ceph-fuse-18.0.0-3905.g1faa3533.el8.x8 3.0 MB/s | 847 kB 00:00 2023-12-14T18:15:49.220 INFO:teuthology.orchestra.run.smithi097.stdout:Total download size: 205 M 2023-12-14T18:15:49.220 INFO:teuthology.orchestra.run.smithi097.stdout:Downloading Packages: 2023-12-14T18:15:49.311 INFO:teuthology.orchestra.run.smithi093.stdout:(3/145): ceph-base-18.0.0-3905.g1faa3533.el8.x8 11 MB/s | 5.3 MB 00:00 2023-12-14T18:15:49.328 INFO:teuthology.orchestra.run.smithi093.stdout:(4/145): ceph-immutable-object-cache-18.0.0-390 1.5 MB/s | 165 kB 00:00 2023-12-14T18:15:49.479 INFO:teuthology.orchestra.run.smithi093.stdout:(5/145): ceph-mds-18.0.0-3905.g1faa3533.el8.x86 14 MB/s | 2.3 MB 00:00 2023-12-14T18:15:49.588 INFO:teuthology.orchestra.run.smithi160.stdout:Transaction test succeeded. 2023-12-14T18:15:49.598 INFO:teuthology.orchestra.run.smithi160.stdout:Running transaction 2023-12-14T18:15:49.620 INFO:teuthology.orchestra.run.smithi093.stdout:(6/145): ceph-mgr-18.0.0-3905.g1faa3533.el8.x86 5.3 MB/s | 1.5 MB 00:00 2023-12-14T18:15:49.756 INFO:teuthology.orchestra.run.smithi097.stdout:(1/145): ceph-18.0.0-3905.g1faa3533.el8.x86_64. 21 kB/s | 6.4 kB 00:00 2023-12-14T18:15:49.805 INFO:teuthology.orchestra.run.smithi093.stdout:(7/145): ceph-mon-18.0.0-3905.g1faa3533.el8.x86 14 MB/s | 4.6 MB 00:00 2023-12-14T18:15:50.623 INFO:teuthology.orchestra.run.smithi097.stdout:(2/145): ceph-base-18.0.0-3905.g1faa3533.el8.x8 4.6 MB/s | 5.3 MB 00:01 2023-12-14T18:15:50.723 INFO:teuthology.orchestra.run.smithi097.stdout:(3/145): ceph-immutable-object-cache-18.0.0-390 1.6 MB/s | 165 kB 00:00 2023-12-14T18:15:51.240 INFO:teuthology.orchestra.run.smithi097.stdout:(4/145): ceph-mds-18.0.0-3905.g1faa3533.el8.x86 4.4 MB/s | 2.3 MB 00:00 2023-12-14T18:15:51.298 INFO:teuthology.orchestra.run.smithi097.stdout:(5/145): ceph-fuse-18.0.0-3905.g1faa3533.el8.x8 549 kB/s | 847 kB 00:01 2023-12-14T18:15:51.381 INFO:teuthology.orchestra.run.smithi093.stdout:(8/145): ceph-radosgw-18.0.0-3905.g1faa3533.el8 8.3 MB/s | 13 MB 00:01 2023-12-14T18:15:51.480 INFO:teuthology.orchestra.run.smithi093.stdout:(9/145): ceph-selinux-18.0.0-3905.g1faa3533.el8 243 kB/s | 24 kB 00:00 2023-12-14T18:15:51.499 INFO:teuthology.orchestra.run.smithi097.stdout:(6/145): ceph-mgr-18.0.0-3905.g1faa3533.el8.x86 6.0 MB/s | 1.5 MB 00:00 2023-12-14T18:15:52.341 INFO:teuthology.orchestra.run.smithi097.stdout:(7/145): ceph-common-18.0.0-3905.g1faa3533.el8. 7.5 MB/s | 22 MB 00:02 2023-12-14T18:15:53.629 INFO:teuthology.orchestra.run.smithi093.stdout:(10/145): ceph-osd-18.0.0-3905.g1faa3533.el8.x8 4.4 MB/s | 18 MB 00:04 2023-12-14T18:15:53.730 INFO:teuthology.orchestra.run.smithi093.stdout:(11/145): cephfs-mirror-18.0.0-3905.g1faa3533.e 2.5 MB/s | 258 kB 00:00 2023-12-14T18:15:53.797 INFO:teuthology.orchestra.run.smithi093.stdout:(12/145): libcephfs-devel-18.0.0-3905.g1faa3533 465 kB/s | 31 kB 00:00 2023-12-14T18:15:53.891 INFO:teuthology.orchestra.run.smithi097.stdout:(8/145): ceph-radosgw-18.0.0-3905.g1faa3533.el8 8.4 MB/s | 13 MB 00:01 2023-12-14T18:15:53.974 INFO:teuthology.orchestra.run.smithi097.stdout:(9/145): ceph-selinux-18.0.0-3905.g1faa3533.el8 292 kB/s | 24 kB 00:00 2023-12-14T18:15:53.989 INFO:teuthology.orchestra.run.smithi093.stdout:(13/145): libcephfs2-18.0.0-3905.g1faa3533.el8. 3.9 MB/s | 771 kB 00:00 2023-12-14T18:15:54.118 INFO:teuthology.orchestra.run.smithi093.stdout:(14/145): libcephsqlite-18.0.0-3905.g1faa3533.e 1.0 MB/s | 137 kB 00:00 2023-12-14T18:15:54.223 INFO:teuthology.orchestra.run.smithi093.stdout:(15/145): librados-devel-18.0.0-3905.g1faa3533. 1.3 MB/s | 139 kB 00:00 2023-12-14T18:15:54.414 INFO:teuthology.orchestra.run.smithi093.stdout:(16/145): libradosstriper1-18.0.0-3905.g1faa353 2.4 MB/s | 473 kB 00:00 2023-12-14T18:15:54.708 INFO:teuthology.orchestra.run.smithi097.stdout:(10/145): ceph-osd-18.0.0-3905.g1faa3533.el8.x8 5.5 MB/s | 18 MB 00:03 2023-12-14T18:15:54.808 INFO:teuthology.orchestra.run.smithi097.stdout:(11/145): cephfs-mirror-18.0.0-3905.g1faa3533.e 2.5 MB/s | 258 kB 00:00 2023-12-14T18:15:54.900 INFO:teuthology.orchestra.run.smithi097.stdout:(12/145): libcephfs-devel-18.0.0-3905.g1faa3533 338 kB/s | 31 kB 00:00 2023-12-14T18:15:55.226 INFO:teuthology.orchestra.run.smithi097.stdout:(13/145): libcephfs2-18.0.0-3905.g1faa3533.el8. 2.3 MB/s | 771 kB 00:00 2023-12-14T18:15:55.306 INFO:teuthology.orchestra.run.smithi160.stdout: Preparing : 1/1 2023-12-14T18:15:55.348 INFO:teuthology.orchestra.run.smithi097.stdout:(14/145): libcephsqlite-18.0.0-3905.g1faa3533.e 1.1 MB/s | 137 kB 00:00 2023-12-14T18:15:55.443 INFO:teuthology.orchestra.run.smithi097.stdout:(15/145): librados-devel-18.0.0-3905.g1faa3533. 1.4 MB/s | 139 kB 00:00 2023-12-14T18:15:55.553 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/1 2023-12-14T18:15:55.609 INFO:teuthology.orchestra.run.smithi097.stdout:(16/145): libradosstriper1-18.0.0-3905.g1faa353 2.8 MB/s | 473 kB 00:00 2023-12-14T18:15:55.998 INFO:teuthology.orchestra.run.smithi093.stdout:(17/145): ceph-test-18.0.0-3905.g1faa3533.el8.x 9.0 MB/s | 41 MB 00:04 2023-12-14T18:15:56.073 INFO:teuthology.orchestra.run.smithi093.stdout:(18/145): librgw2-18.0.0-3905.g1faa3533.el8.x86 3.9 MB/s | 6.5 MB 00:01 2023-12-14T18:15:56.099 INFO:teuthology.orchestra.run.smithi093.stdout:(19/145): python3-ceph-argparse-18.0.0-3905.g1f 447 kB/s | 45 kB 00:00 2023-12-14T18:15:56.140 INFO:teuthology.orchestra.run.smithi093.stdout:(20/145): python3-ceph-common-18.0.0-3905.g1faa 1.7 MB/s | 119 kB 00:00 2023-12-14T18:15:56.174 INFO:teuthology.orchestra.run.smithi093.stdout:(21/145): python3-cephfs-18.0.0-3905.g1faa3533. 2.6 MB/s | 197 kB 00:00 2023-12-14T18:15:56.248 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:15:56.266 INFO:teuthology.orchestra.run.smithi093.stdout:(22/145): python3-rbd-18.0.0-3905.g1faa3533.el8 3.9 MB/s | 364 kB 00:00 2023-12-14T18:15:56.291 INFO:teuthology.orchestra.run.smithi093.stdout:(23/145): python3-rados-18.0.0-3905.g1faa3533.e 2.4 MB/s | 370 kB 00:00 2023-12-14T18:15:56.317 INFO:teuthology.orchestra.run.smithi093.stdout:(24/145): python3-rgw-18.0.0-3905.g1faa3533.el8 2.1 MB/s | 107 kB 00:00 2023-12-14T18:15:56.335 INFO:teuthology.orchestra.run.smithi097.stdout:(17/145): librgw2-18.0.0-3905.g1faa3533.el8.x86 9.0 MB/s | 6.5 MB 00:00 2023-12-14T18:15:56.342 INFO:teuthology.orchestra.run.smithi093.stdout:(25/145): rbd-fuse-18.0.0-3905.g1faa3533.el8.x8 1.8 MB/s | 94 kB 00:00 2023-12-14T18:15:56.393 INFO:teuthology.orchestra.run.smithi097.stdout:(18/145): python3-ceph-argparse-18.0.0-3905.g1f 771 kB/s | 45 kB 00:00 2023-12-14T18:15:56.425 INFO:teuthology.orchestra.run.smithi093.stdout:(26/145): rbd-nbd-18.0.0-3905.g1faa3533.el8.x86 2.1 MB/s | 179 kB 00:00 2023-12-14T18:15:56.477 INFO:teuthology.orchestra.run.smithi097.stdout:(19/145): python3-ceph-common-18.0.0-3905.g1faa 1.4 MB/s | 119 kB 00:00 2023-12-14T18:15:56.503 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:15:56.509 INFO:teuthology.orchestra.run.smithi093.stdout:(27/145): ceph-grafana-dashboards-18.0.0-3905.g 277 kB/s | 23 kB 00:00 2023-12-14T18:15:56.586 INFO:teuthology.orchestra.run.smithi097.stdout:(20/145): ceph-mon-18.0.0-3905.g1faa3533.el8.x8 892 kB/s | 4.6 MB 00:05 2023-12-14T18:15:56.593 INFO:teuthology.orchestra.run.smithi093.stdout:(28/145): ceph-mgr-cephadm-18.0.0-3905.g1faa353 1.5 MB/s | 127 kB 00:00 2023-12-14T18:15:56.619 INFO:teuthology.orchestra.run.smithi097.stdout:(21/145): python3-cephfs-18.0.0-3905.g1faa3533. 1.4 MB/s | 197 kB 00:00 2023-12-14T18:15:56.659 INFO:teuthology.orchestra.run.smithi093.stdout:(29/145): rbd-mirror-18.0.0-3905.g1faa3533.el8. 10 MB/s | 3.4 MB 00:00 2023-12-14T18:15:56.694 INFO:teuthology.orchestra.run.smithi097.stdout:(22/145): python3-rbd-18.0.0-3905.g1faa3533.el8 4.8 MB/s | 364 kB 00:00 2023-12-14T18:15:56.742 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/161 2023-12-14T18:15:56.786 INFO:teuthology.orchestra.run.smithi097.stdout:(23/145): python3-rgw-18.0.0-3905.g1faa3533.el8 1.1 MB/s | 107 kB 00:00 2023-12-14T18:15:56.887 INFO:teuthology.orchestra.run.smithi097.stdout:(24/145): rbd-fuse-18.0.0-3905.g1faa3533.el8.x8 941 kB/s | 94 kB 00:00 2023-12-14T18:15:56.911 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:15:56.912 INFO:teuthology.orchestra.run.smithi097.stdout:(25/145): python3-rados-18.0.0-3905.g1faa3533.e 1.1 MB/s | 370 kB 00:00 2023-12-14T18:15:56.976 INFO:teuthology.orchestra.run.smithi093.stdout:(30/145): ceph-mgr-dashboard-18.0.0-3905.g1faa3 4.5 MB/s | 1.7 MB 00:00 2023-12-14T18:15:57.017 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:15:57.077 INFO:teuthology.orchestra.run.smithi093.stdout:(31/145): ceph-mgr-modules-core-18.0.0-3905.g1f 2.3 MB/s | 240 kB 00:00 2023-12-14T18:15:57.129 INFO:teuthology.orchestra.run.smithi097.stdout:(26/145): rbd-nbd-18.0.0-3905.g1faa3533.el8.x86 829 kB/s | 179 kB 00:00 2023-12-14T18:15:57.182 INFO:teuthology.orchestra.run.smithi093.stdout:(32/145): ceph-mgr-rook-18.0.0-3905.g1faa3533.e 451 kB/s | 47 kB 00:00 2023-12-14T18:15:57.211 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:15:57.238 INFO:teuthology.orchestra.run.smithi097.stdout:(27/145): ceph-grafana-dashboards-18.0.0-3905.g 213 kB/s | 23 kB 00:00 2023-12-14T18:15:57.269 INFO:teuthology.orchestra.run.smithi093.stdout:(33/145): ceph-prometheus-alerts-18.0.0-3905.g1 168 kB/s | 15 kB 00:00 2023-12-14T18:15:57.315 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:15:57.369 INFO:teuthology.orchestra.run.smithi093.stdout:(34/145): ceph-volume-18.0.0-3905.g1faa3533.el8 2.6 MB/s | 265 kB 00:00 2023-12-14T18:15:57.430 INFO:teuthology.orchestra.run.smithi097.stdout:(28/145): ceph-mgr-cephadm-18.0.0-3905.g1faa353 665 kB/s | 127 kB 00:00 2023-12-14T18:15:57.497 INFO:teuthology.orchestra.run.smithi097.stdout:(29/145): rbd-mirror-18.0.0-3905.g1faa3533.el8. 5.7 MB/s | 3.4 MB 00:00 2023-12-14T18:15:57.503 INFO:teuthology.orchestra.run.smithi093.stdout:(35/145): ceph-mgr-diskprediction-local-18.0.0- 8.8 MB/s | 7.4 MB 00:00 2023-12-14T18:15:57.526 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/161 2023-12-14T18:15:57.528 INFO:teuthology.orchestra.run.smithi093.stdout:(36/145): cephadm-18.0.0-3905.g1faa3533.el8.noa 526 kB/s | 83 kB 00:00 2023-12-14T18:15:57.578 INFO:teuthology.orchestra.run.smithi093.stdout:(37/145): cephfs-top-18.0.0-3905.g1faa3533.el8. 239 kB/s | 18 kB 00:00 2023-12-14T18:15:57.711 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/161 2023-12-14T18:15:57.782 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/161 2023-12-14T18:15:57.912 INFO:teuthology.orchestra.run.smithi093.stdout:(38/145): liboath-2.6.2-3.el8.x86_64.rpm 179 kB/s | 59 kB 00:00 2023-12-14T18:15:57.937 INFO:teuthology.orchestra.run.smithi093.stdout:(39/145): gperftools-libs-2.7-9.el8.x86_64.rpm 749 kB/s | 306 kB 00:00 2023-12-14T18:15:58.011 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/161 2023-12-14T18:15:58.088 INFO:teuthology.orchestra.run.smithi093.stdout:(40/145): libunwind-1.3.1-3.el8.x86_64.rpm 429 kB/s | 75 kB 00:00 2023-12-14T18:15:58.171 INFO:teuthology.orchestra.run.smithi093.stdout:(41/145): python3-asyncssh-2.7.0-2.el8.noarch.r 2.0 MB/s | 476 kB 00:00 2023-12-14T18:15:58.197 INFO:teuthology.orchestra.run.smithi093.stdout:(42/145): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 401 kB/s | 44 kB 00:00 2023-12-14T18:15:58.214 INFO:teuthology.orchestra.run.smithi097.stdout:(30/145): ceph-test-18.0.0-3905.g1faa3533.el8.x 9.6 MB/s | 41 MB 00:04 2023-12-14T18:15:58.236 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:15:58.272 INFO:teuthology.orchestra.run.smithi093.stdout:(43/145): python3-beautifulsoup4-4.6.3-2.el8.1. 1.8 MB/s | 185 kB 00:00 2023-12-14T18:15:58.273 INFO:teuthology.orchestra.run.smithi097.stdout:(31/145): ceph-mgr-modules-core-18.0.0-3905.g1f 4.1 MB/s | 240 kB 00:00 2023-12-14T18:15:58.297 INFO:teuthology.orchestra.run.smithi093.stdout:(44/145): python3-cachetools-3.1.1-4.el8.noarch 329 kB/s | 33 kB 00:00 2023-12-14T18:15:58.385 INFO:teuthology.orchestra.run.smithi093.stdout:(45/145): python3-certifi-2018.10.15-7.el8.noar 144 kB/s | 16 kB 00:00 2023-12-14T18:15:58.398 INFO:teuthology.orchestra.run.smithi097.stdout:(32/145): ceph-mgr-diskprediction-local-18.0.0- 8.2 MB/s | 7.4 MB 00:00 2023-12-14T18:15:58.423 INFO:teuthology.orchestra.run.smithi097.stdout:(33/145): ceph-mgr-rook-18.0.0-3905.g1faa3533.e 316 kB/s | 47 kB 00:00 2023-12-14T18:15:58.448 INFO:teuthology.orchestra.run.smithi097.stdout:(34/145): ceph-prometheus-alerts-18.0.0-3905.g1 290 kB/s | 15 kB 00:00 2023-12-14T18:15:58.464 INFO:teuthology.orchestra.run.smithi093.stdout:(46/145): python3-cheroot-8.5.2-1.el8.noarch.rp 1.0 MB/s | 173 kB 00:00 2023-12-14T18:15:58.489 INFO:teuthology.orchestra.run.smithi093.stdout:(47/145): python3-cherrypy-18.4.0-1.el8.noarch. 3.6 MB/s | 384 kB 00:00 2023-12-14T18:15:58.498 INFO:teuthology.orchestra.run.smithi097.stdout:(35/145): ceph-volume-18.0.0-3905.g1faa3533.el8 3.4 MB/s | 265 kB 00:00 2023-12-14T18:15:58.532 INFO:teuthology.orchestra.run.smithi097.stdout:(36/145): ceph-mgr-dashboard-18.0.0-3905.g1faa3 1.6 MB/s | 1.7 MB 00:01 2023-12-14T18:15:58.556 INFO:teuthology.orchestra.run.smithi093.stdout:(48/145): python3-google-auth-1.1.1-10.el8.noar 884 kB/s | 81 kB 00:00 2023-12-14T18:15:58.557 INFO:teuthology.orchestra.run.smithi097.stdout:(37/145): cephfs-top-18.0.0-3905.g1faa3533.el8. 307 kB/s | 18 kB 00:00 2023-12-14T18:15:58.574 INFO:teuthology.orchestra.run.smithi097.stdout:(38/145): cephadm-18.0.0-3905.g1faa3533.el8.noa 664 kB/s | 83 kB 00:00 2023-12-14T18:15:58.590 INFO:teuthology.orchestra.run.smithi093.stdout:(49/145): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2023-12-14T18:15:58.644 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:15:58.657 INFO:teuthology.orchestra.run.smithi093.stdout:(50/145): python3-jaraco-functools-2.0-4.el8.no 175 kB/s | 18 kB 00:00 2023-12-14T18:15:58.774 INFO:teuthology.orchestra.run.smithi093.stdout:(51/145): python3-kubernetes-11.0.0-6.el8.noarc 6.4 MB/s | 1.2 MB 00:00 2023-12-14T18:15:58.799 INFO:teuthology.orchestra.run.smithi093.stdout:(52/145): python3-logutils-0.3.5-11.el8.noarch. 344 kB/s | 49 kB 00:00 2023-12-14T18:15:58.866 INFO:teuthology.orchestra.run.smithi097.stdout:(39/145): liboath-2.6.2-3.el8.x86_64.rpm 190 kB/s | 59 kB 00:00 2023-12-14T18:15:58.878 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:15:58.891 INFO:teuthology.orchestra.run.smithi093.stdout:(53/145): python3-more-itertools-7.2.0-3.el8.no 502 kB/s | 59 kB 00:00 2023-12-14T18:15:58.891 INFO:teuthology.orchestra.run.smithi097.stdout:(40/145): libunwind-1.3.1-3.el8.x86_64.rpm 237 kB/s | 75 kB 00:00 2023-12-14T18:15:58.916 INFO:teuthology.orchestra.run.smithi093.stdout:(54/145): python3-natsort-7.1.1-2.el8.noarch.rp 515 kB/s | 60 kB 00:00 2023-12-14T18:15:58.967 INFO:teuthology.orchestra.run.smithi097.stdout:(41/145): gperftools-libs-2.7-9.el8.x86_64.rpm 705 kB/s | 306 kB 00:00 2023-12-14T18:15:58.992 INFO:teuthology.orchestra.run.smithi093.stdout:(55/145): python3-pecan-1.3.2-9.el8.noarch.rpm 2.8 MB/s | 283 kB 00:00 2023-12-14T18:15:58.992 INFO:teuthology.orchestra.run.smithi097.stdout:(42/145): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 435 kB/s | 44 kB 00:00 2023-12-14T18:15:59.017 INFO:teuthology.orchestra.run.smithi093.stdout:(56/145): python3-portend-2.6-1.el8.noarch.rpm 155 kB/s | 16 kB 00:00 2023-12-14T18:15:59.077 INFO:teuthology.orchestra.run.smithi093.stdout:(57/145): python3-repoze-lru-0.7-6.el8.noarch.r 402 kB/s | 34 kB 00:00 2023-12-14T18:15:59.101 INFO:teuthology.orchestra.run.smithi097.stdout:(43/145): python3-beautifulsoup4-4.6.3-2.el8.1. 1.4 MB/s | 185 kB 00:00 2023-12-14T18:15:59.126 INFO:teuthology.orchestra.run.smithi097.stdout:(44/145): python3-cachetools-3.1.1-4.el8.noarch 247 kB/s | 33 kB 00:00 2023-12-14T18:15:59.151 INFO:teuthology.orchestra.run.smithi097.stdout:(45/145): python3-asyncssh-2.7.0-2.el8.noarch.r 1.6 MB/s | 476 kB 00:00 2023-12-14T18:15:59.159 INFO:teuthology.orchestra.run.smithi093.stdout:(58/145): python3-routes-2.4.1-12.el8.noarch.rp 1.4 MB/s | 196 kB 00:00 2023-12-14T18:15:59.176 INFO:teuthology.orchestra.run.smithi093.stdout:(59/145): python3-rsa-4.9-2.el8.noarch.rpm 642 kB/s | 64 kB 00:00 2023-12-14T18:15:59.176 INFO:teuthology.orchestra.run.smithi097.stdout:(46/145): python3-certifi-2018.10.15-7.el8.noar 216 kB/s | 16 kB 00:00 2023-12-14T18:15:59.251 INFO:teuthology.orchestra.run.smithi093.stdout:(60/145): python3-simplegeneric-0.8.1-17.el8.no 204 kB/s | 19 kB 00:00 2023-12-14T18:15:59.260 INFO:teuthology.orchestra.run.smithi097.stdout:(47/145): python3-cherrypy-18.4.0-1.el8.noarch. 3.5 MB/s | 384 kB 00:00 2023-12-14T18:15:59.277 INFO:teuthology.orchestra.run.smithi093.stdout:(61/145): python3-singledispatch-3.4.0.3-18.el8 238 kB/s | 24 kB 00:00 2023-12-14T18:15:59.285 INFO:teuthology.orchestra.run.smithi097.stdout:(48/145): python3-google-auth-1.1.1-10.el8.noar 748 kB/s | 81 kB 00:00 2023-12-14T18:15:59.307 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:15:59.311 INFO:teuthology.orchestra.run.smithi097.stdout:(49/145): python3-cheroot-8.5.2-1.el8.noarch.rp 936 kB/s | 173 kB 00:00 2023-12-14T18:15:59.367 INFO:teuthology.orchestra.run.smithi097.stdout:(50/145): python3-jaraco-6.2-6.el8.noarch.rpm 104 kB/s | 11 kB 00:00 2023-12-14T18:15:59.486 INFO:teuthology.orchestra.run.smithi097.stdout:(51/145): python3-jaraco-functools-2.0-4.el8.no 88 kB/s | 18 kB 00:00 2023-12-14T18:15:59.512 INFO:teuthology.orchestra.run.smithi097.stdout:(52/145): python3-logutils-0.3.5-11.el8.noarch. 338 kB/s | 49 kB 00:00 2023-12-14T18:15:59.518 INFO:teuthology.orchestra.run.smithi093.stdout:(62/145): ceph-common-18.0.0-3905.g1faa3533.el8 2.0 MB/s | 22 MB 00:10 2023-12-14T18:15:59.519 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:15:59.544 INFO:teuthology.orchestra.run.smithi093.stdout:(63/145): python3-tempora-1.14.1-5.el8.noarch.r 100 kB/s | 29 kB 00:00 2023-12-14T18:15:59.595 INFO:teuthology.orchestra.run.smithi093.stdout:(64/145): python3-trustme-0.6.0-4.el8.noarch.rp 86 kB/s | 27 kB 00:00 2023-12-14T18:15:59.612 INFO:teuthology.orchestra.run.smithi097.stdout:(53/145): python3-more-itertools-7.2.0-3.el8.no 469 kB/s | 59 kB 00:00 2023-12-14T18:15:59.630 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:15:59.644 INFO:teuthology.orchestra.run.smithi093.stdout:(65/145): python3-webob-1.8.5-1.el8.1.noarch.rp 2.5 MB/s | 251 kB 00:00 2023-12-14T18:15:59.670 INFO:teuthology.orchestra.run.smithi097.stdout:(54/145): python3-kubernetes-11.0.0-6.el8.noarc 3.2 MB/s | 1.2 MB 00:00 2023-12-14T18:15:59.721 INFO:teuthology.orchestra.run.smithi097.stdout:(55/145): python3-natsort-7.1.1-2.el8.noarch.rp 288 kB/s | 60 kB 00:00 2023-12-14T18:15:59.746 INFO:teuthology.orchestra.run.smithi097.stdout:(56/145): python3-portend-2.6-1.el8.noarch.rpm 205 kB/s | 16 kB 00:00 2023-12-14T18:15:59.761 INFO:teuthology.orchestra.run.smithi093.stdout:(66/145): python3-webtest-2.0.33-1.el8.noarch.r 732 kB/s | 85 kB 00:00 2023-12-14T18:15:59.787 INFO:teuthology.orchestra.run.smithi093.stdout:(67/145): python3-waitress-1.4.3-1.el8.noarch.r 954 kB/s | 255 kB 00:00 2023-12-14T18:15:59.788 INFO:teuthology.orchestra.run.smithi097.stdout:(57/145): python3-pecan-1.3.2-9.el8.noarch.rpm 1.6 MB/s | 283 kB 00:00 2023-12-14T18:15:59.814 INFO:teuthology.orchestra.run.smithi097.stdout:(58/145): python3-repoze-lru-0.7-6.el8.noarch.r 365 kB/s | 34 kB 00:00 2023-12-14T18:15:59.839 INFO:teuthology.orchestra.run.smithi097.stdout:(59/145): python3-routes-2.4.1-12.el8.noarch.rp 2.1 MB/s | 196 kB 00:00 2023-12-14T18:15:59.854 INFO:teuthology.orchestra.run.smithi093.stdout:(68/145): python3-zc-lockfile-2.0-2.el8.noarch. 249 kB/s | 23 kB 00:00 2023-12-14T18:15:59.861 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:15:59.864 INFO:teuthology.orchestra.run.smithi097.stdout:(60/145): python3-rsa-4.9-2.el8.noarch.rpm 852 kB/s | 64 kB 00:00 2023-12-14T18:15:59.944 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:15:59.962 INFO:teuthology.orchestra.run.smithi093.stdout:(69/145): xmlstarlet-1.6.1-20.el8.x86_64.rpm 644 kB/s | 70 kB 00:00 2023-12-14T18:16:00.089 INFO:teuthology.orchestra.run.smithi097.stdout:(61/145): python3-singledispatch-3.4.0.3-18.el8 96 kB/s | 24 kB 00:00 2023-12-14T18:16:00.106 INFO:teuthology.orchestra.run.smithi097.stdout:(62/145): python3-simplegeneric-0.8.1-17.el8.no 64 kB/s | 19 kB 00:00 2023-12-14T18:16:00.123 INFO:teuthology.orchestra.run.smithi097.stdout:(63/145): python3-tempora-1.14.1-5.el8.noarch.r 113 kB/s | 29 kB 00:00 2023-12-14T18:16:00.196 INFO:teuthology.orchestra.run.smithi093.stdout:(70/145): python3-websocket-client-0.56.0-5.el8 102 kB/s | 61 kB 00:00 2023-12-14T18:16:00.273 INFO:teuthology.orchestra.run.smithi097.stdout:(64/145): python3-trustme-0.6.0-4.el8.noarch.rp 150 kB/s | 27 kB 00:00 2023-12-14T18:16:00.294 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:16:00.321 INFO:teuthology.orchestra.run.smithi093.stdout:(71/145): thrift-0.13.0-2.el8.x86_64.rpm 3.2 MB/s | 1.7 MB 00:00 2023-12-14T18:16:00.338 INFO:teuthology.orchestra.run.smithi093.stdout:(72/145): libverto-devel-0.3.0-5.el8.x86_64.rpm 48 kB/s | 18 kB 00:00 2023-12-14T18:16:00.363 INFO:teuthology.orchestra.run.smithi093.stdout:(73/145): python3-jwt-1.6.1-2.el8.noarch.rpm 255 kB/s | 43 kB 00:00 2023-12-14T18:16:00.457 INFO:teuthology.orchestra.run.smithi097.stdout:(65/145): python3-waitress-1.4.3-1.el8.noarch.r 766 kB/s | 255 kB 00:00 2023-12-14T18:16:00.471 INFO:teuthology.orchestra.run.smithi093.stdout:(74/145): python3-pycparser-2.14-14.el8.noarch. 856 kB/s | 109 kB 00:00 2023-12-14T18:16:00.505 INFO:teuthology.orchestra.run.smithi093.stdout:(75/145): python3-requests-oauthlib-1.0.0-1.el8 301 kB/s | 43 kB 00:00 2023-12-14T18:16:00.524 INFO:teuthology.orchestra.run.smithi097.stdout:(66/145): python3-webob-1.8.5-1.el8.1.noarch.rp 627 kB/s | 251 kB 00:00 2023-12-14T18:16:00.531 INFO:teuthology.orchestra.run.smithi093.stdout:(76/145): python3-oauthlib-2.1.0-1.el8.noarch.r 740 kB/s | 155 kB 00:00 2023-12-14T18:16:00.568 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:00.572 INFO:teuthology.orchestra.run.smithi093.stdout:(77/145): libaio-devel-0.3.112-1.el8.x86_64.rpm 185 kB/s | 19 kB 00:00 2023-12-14T18:16:00.631 INFO:teuthology.orchestra.run.smithi093.stdout:(78/145): pcre2-devel-10.32-2.el8.x86_64.rpm 5.9 MB/s | 605 kB 00:00 2023-12-14T18:16:00.656 INFO:teuthology.orchestra.run.smithi093.stdout:(79/145): libzstd-devel-1.4.4-1.el8.x86_64.rpm 291 kB/s | 44 kB 00:00 2023-12-14T18:16:00.694 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:00.733 INFO:teuthology.orchestra.run.smithi093.stdout:(80/145): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.4 MB/s | 229 kB 00:00 2023-12-14T18:16:00.807 INFO:teuthology.orchestra.run.smithi093.stdout:(81/145): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.2 MB/s | 220 kB 00:00 2023-12-14T18:16:00.849 INFO:teuthology.orchestra.run.smithi093.stdout:(82/145): libselinux-devel-2.9-5.el8.x86_64.rpm 1.0 MB/s | 200 kB 00:00 2023-12-14T18:16:00.890 INFO:teuthology.orchestra.run.smithi093.stdout:(83/145): keyutils-libs-devel-1.5.10-9.el8.x86_ 305 kB/s | 48 kB 00:00 2023-12-14T18:16:00.918 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libpq-13.11-1.el8.x86_64 14/161 2023-12-14T18:16:00.991 INFO:teuthology.orchestra.run.smithi093.stdout:(84/145): zlib-devel-1.2.11-18.el8_5.x86_64.rpm 407 kB/s | 58 kB 00:00 2023-12-14T18:16:01.016 INFO:teuthology.orchestra.run.smithi093.stdout:(85/145): libcom_err-devel-1.45.6-4.el8.x86_64. 308 kB/s | 39 kB 00:00 2023-12-14T18:16:01.042 INFO:teuthology.orchestra.run.smithi093.stdout:(86/145): libsepol-devel-2.9-3.el8.x86_64.rpm 371 kB/s | 87 kB 00:00 2023-12-14T18:16:01.057 INFO:teuthology.orchestra.run.smithi097.stdout:(67/145): python3-websocket-client-0.56.0-5.el8 78 kB/s | 61 kB 00:00 2023-12-14T18:16:01.150 INFO:teuthology.orchestra.run.smithi097.stdout:(68/145): python3-zc-lockfile-2.0-2.el8.noarch. 37 kB/s | 23 kB 00:00 2023-12-14T18:16:01.150 INFO:teuthology.orchestra.run.smithi093.stdout:(87/145): openssl-devel-1.1.1k-9.el8_7.x86_64.r 15 MB/s | 2.3 MB 00:00 2023-12-14T18:16:01.175 INFO:teuthology.orchestra.run.smithi097.stdout:(69/145): python3-webtest-2.0.33-1.el8.noarch.r 119 kB/s | 85 kB 00:00 2023-12-14T18:16:01.189 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/161 2023-12-14T18:16:01.200 INFO:teuthology.orchestra.run.smithi097.stdout:(70/145): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.4 MB/s | 70 kB 00:00 2023-12-14T18:16:01.214 INFO:teuthology.orchestra.run.smithi093.stdout:(88/145): libgfortran-8.5.0-20.el8.x86_64.rpm 3.7 MB/s | 645 kB 00:00 2023-12-14T18:16:01.251 INFO:teuthology.orchestra.run.smithi093.stdout:(89/145): fuse-2.9.7-17.el8.x86_64.rpm 355 kB/s | 83 kB 00:00 2023-12-14T18:16:01.276 INFO:teuthology.orchestra.run.smithi093.stdout:(90/145): libquadmath-8.5.0-20.el8.x86_64.rpm 1.3 MB/s | 172 kB 00:00 2023-12-14T18:16:01.325 INFO:teuthology.orchestra.run.smithi097.stdout:(71/145): python3-jsonpointer-1.10-11.el8.noarc 133 kB/s | 20 kB 00:00 2023-12-14T18:16:01.343 INFO:teuthology.orchestra.run.smithi093.stdout:(91/145): libuuid-devel-2.32.1-43.el8.x86_64.rp 778 kB/s | 100 kB 00:00 2023-12-14T18:16:01.359 INFO:teuthology.orchestra.run.smithi093.stdout:(92/145): python3-cffi-1.11.5-6.el8.x86_64.rpm 2.1 MB/s | 238 kB 00:00 2023-12-14T18:16:01.367 INFO:teuthology.orchestra.run.smithi097.stdout:(72/145): python3-prettytable-0.7.2-14.el8.noar 272 kB/s | 44 kB 00:00 2023-12-14T18:16:01.380 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2023-12-14T18:16:01.401 INFO:teuthology.orchestra.run.smithi093.stdout:(93/145): python3-cryptography-3.2.1-6.el8.x86_ 4.4 MB/s | 559 kB 00:00 2023-12-14T18:16:01.451 INFO:teuthology.orchestra.run.smithi097.stdout:(73/145): thrift-0.13.0-2.el8.x86_64.rpm 4.4 MB/s | 1.7 MB 00:00 2023-12-14T18:16:01.476 INFO:teuthology.orchestra.run.smithi097.stdout:(74/145): python3-pyasn1-modules-0.3.7-6.el8.no 732 kB/s | 110 kB 00:00 2023-12-14T18:16:01.485 INFO:teuthology.orchestra.run.smithi093.stdout:(94/145): elfutils-libelf-devel-0.189-3.el8.x86 742 kB/s | 62 kB 00:00 2023-12-14T18:16:01.502 INFO:teuthology.orchestra.run.smithi093.stdout:(95/145): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 457 kB/s | 73 kB 00:00 2023-12-14T18:16:01.526 INFO:teuthology.orchestra.run.smithi097.stdout:(75/145): python3-jsonpatch-1.21-2.el8.noarch.r 167 kB/s | 27 kB 00:00 2023-12-14T18:16:01.527 INFO:teuthology.orchestra.run.smithi093.stdout:(96/145): NetworkManager-initscripts-updown-1.4 858 kB/s | 143 kB 00:00 2023-12-14T18:16:01.577 INFO:teuthology.orchestra.run.smithi097.stdout:(76/145): python3-werkzeug-0.12.2-4.el8.noarch. 4.7 MB/s | 457 kB 00:00 2023-12-14T18:16:01.611 INFO:teuthology.orchestra.run.smithi093.stdout:(97/145): krb5-devel-1.18.2-26.el8_9.x86_64.rpm 5.1 MB/s | 562 kB 00:00 2023-12-14T18:16:01.635 INFO:teuthology.orchestra.run.smithi097.stdout:(77/145): python3-pyasn1-0.3.7-6.el8.noarch.rpm 687 kB/s | 126 kB 00:00 2023-12-14T18:16:01.653 INFO:teuthology.orchestra.run.smithi093.stdout:(98/145): librabbitmq-0.9.0-4.el8.x86_64.rpm 281 kB/s | 47 kB 00:00 2023-12-14T18:16:01.669 INFO:teuthology.orchestra.run.smithi097.stdout:(78/145): flex-2.6.1-9.el8.x86_64.rpm 3.4 MB/s | 320 kB 00:00 2023-12-14T18:16:01.694 INFO:teuthology.orchestra.run.smithi097.stdout:(79/145): bison-3.0.4-10.el8.x86_64.rpm 4.0 MB/s | 688 kB 00:00 2023-12-14T18:16:01.728 INFO:teuthology.orchestra.run.smithi093.stdout:(99/145): python3-prettytable-0.7.2-14.el8.noar 376 kB/s | 44 kB 00:00 2023-12-14T18:16:01.762 INFO:teuthology.orchestra.run.smithi093.stdout:(100/145): python3-jsonpointer-1.10-11.el8.noar 85 kB/s | 20 kB 00:00 2023-12-14T18:16:01.786 INFO:teuthology.orchestra.run.smithi097.stdout:(80/145): libtool-2.4.6-25.el8.x86_64.rpm 4.6 MB/s | 709 kB 00:00 2023-12-14T18:16:01.787 INFO:teuthology.orchestra.run.smithi093.stdout:(101/145): python3-pyasn1-modules-0.3.7-6.el8.n 822 kB/s | 110 kB 00:00 2023-12-14T18:16:01.828 INFO:teuthology.orchestra.run.smithi097.stdout:(81/145): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.9 MB/s | 259 kB 00:00 2023-12-14T18:16:01.835 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/161 2023-12-14T18:16:01.870 INFO:teuthology.orchestra.run.smithi097.stdout:(82/145): python3-markupsafe-0.23-19.el8.x86_64 193 kB/s | 39 kB 00:00 2023-12-14T18:16:01.904 INFO:teuthology.orchestra.run.smithi093.stdout:(102/145): python3-werkzeug-0.12.2-4.el8.noarch 3.8 MB/s | 457 kB 00:00 2023-12-14T18:16:01.937 INFO:teuthology.orchestra.run.smithi097.stdout:(83/145): uuid-1.6.2-43.el8.x86_64.rpm 589 kB/s | 64 kB 00:00 2023-12-14T18:16:01.954 INFO:teuthology.orchestra.run.smithi093.stdout:(103/145): python3-jsonpatch-1.21-2.el8.noarch. 117 kB/s | 27 kB 00:00 2023-12-14T18:16:01.963 INFO:teuthology.orchestra.run.smithi097.stdout:(84/145): oniguruma-6.8.2-2.el8.x86_64.rpm 1.0 MB/s | 187 kB 00:00 2023-12-14T18:16:01.979 INFO:teuthology.orchestra.run.smithi093.stdout:(104/145): python3-pyasn1-0.3.7-6.el8.noarch.rp 581 kB/s | 126 kB 00:00 2023-12-14T18:16:02.030 INFO:teuthology.orchestra.run.smithi093.stdout:(105/145): bison-3.0.4-10.el8.x86_64.rpm 5.4 MB/s | 688 kB 00:00 2023-12-14T18:16:02.081 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 18/161 2023-12-14T18:16:02.105 INFO:teuthology.orchestra.run.smithi093.stdout:(106/145): flex-2.6.1-9.el8.x86_64.rpm 2.1 MB/s | 320 kB 00:00 2023-12-14T18:16:02.121 INFO:teuthology.orchestra.run.smithi097.stdout:(85/145): python3-pyOpenSSL-19.0.0-1.el8.noarch 413 kB/s | 103 kB 00:00 2023-12-14T18:16:02.138 INFO:teuthology.orchestra.run.smithi093.stdout:(107/145): libtool-2.4.6-25.el8.x86_64.rpm 4.4 MB/s | 709 kB 00:00 2023-12-14T18:16:02.197 INFO:teuthology.orchestra.run.smithi093.stdout:(108/145): python3-markupsafe-0.23-19.el8.x86_6 232 kB/s | 39 kB 00:00 2023-12-14T18:16:02.235 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:16:02.247 INFO:teuthology.orchestra.run.smithi097.stdout:(86/145): python3-babel-2.5.1-7.el8.noarch.rpm 17 MB/s | 4.8 MB 00:00 2023-12-14T18:16:02.272 INFO:teuthology.orchestra.run.smithi093.stdout:(109/145): oniguruma-6.8.2-2.el8.x86_64.rpm 1.4 MB/s | 187 kB 00:00 2023-12-14T18:16:02.280 INFO:teuthology.orchestra.run.smithi097.stdout:(87/145): python3-jinja2-2.10.1-3.el8.noarch.rp 3.3 MB/s | 538 kB 00:00 2023-12-14T18:16:02.298 INFO:teuthology.orchestra.run.smithi093.stdout:(110/145): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2023-12-14T18:16:02.336 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:16:02.339 INFO:teuthology.orchestra.run.smithi093.stdout:(111/145): uuid-1.6.2-43.el8.x86_64.rpm 450 kB/s | 64 kB 00:00 2023-12-14T18:16:02.500 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:16:02.521 INFO:teuthology.orchestra.run.smithi097.stdout:(88/145): python3-scipy-1.0.0-21.module+el8.5.0 23 MB/s | 14 MB 00:00 2023-12-14T18:16:02.548 INFO:teuthology.orchestra.run.smithi093.stdout:(112/145): python3-babel-2.5.1-7.el8.noarch.rpm 23 MB/s | 4.8 MB 00:00 2023-12-14T18:16:02.556 INFO:teuthology.orchestra.run.smithi097.stdout:(89/145): socat-1.7.4.1-1.el8.x86_64.rpm 1.1 MB/s | 323 kB 00:00 2023-12-14T18:16:02.573 INFO:teuthology.orchestra.run.smithi093.stdout:(113/145): python3-pyOpenSSL-19.0.0-1.el8.noarc 347 kB/s | 103 kB 00:00 2023-12-14T18:16:02.584 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:16:02.623 INFO:teuthology.orchestra.run.smithi097.stdout:(90/145): python3-numpy-1.14.3-10.el8.x86_64.rp 9.9 MB/s | 3.7 MB 00:00 2023-12-14T18:16:02.640 INFO:teuthology.orchestra.run.smithi097.stdout:(91/145): python3-numpy-f2py-1.14.3-10.el8.x86_ 1.9 MB/s | 225 kB 00:00 2023-12-14T18:16:02.707 INFO:teuthology.orchestra.run.smithi097.stdout:(92/145): librdkafka-0.11.4-3.el8.x86_64.rpm 2.3 MB/s | 354 kB 00:00 2023-12-14T18:16:02.732 INFO:teuthology.orchestra.run.smithi097.stdout:(93/145): python3-mako-1.0.6-14.el8.noarch.rpm 1.7 MB/s | 157 kB 00:00 2023-12-14T18:16:02.760 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/161 2023-12-14T18:16:02.790 INFO:teuthology.orchestra.run.smithi093.stdout:(114/145): python3-scipy-1.0.0-21.module+el8.5. 28 MB/s | 14 MB 00:00 2023-12-14T18:16:02.824 INFO:teuthology.orchestra.run.smithi093.stdout:(115/145): python3-jinja2-2.10.1-3.el8.noarch.r 1.9 MB/s | 538 kB 00:00 2023-12-14T18:16:02.874 INFO:teuthology.orchestra.run.smithi097.stdout:(94/145): openblas-threads-0.3.15-6.el8.x86_64. 21 MB/s | 5.2 MB 00:00 2023-12-14T18:16:02.905 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/161 2023-12-14T18:16:02.924 INFO:teuthology.orchestra.run.smithi097.stdout:(95/145): postgresql-10.23-2.module+el8.8.0+194 7.8 MB/s | 1.5 MB 00:00 2023-12-14T18:16:02.940 INFO:teuthology.orchestra.run.smithi093.stdout:(116/145): python3-numpy-1.14.3-10.el8.x86_64.r 10 MB/s | 3.7 MB 00:00 2023-12-14T18:16:02.966 INFO:teuthology.orchestra.run.smithi093.stdout:(117/145): socat-1.7.4.1-1.el8.x86_64.rpm 1.8 MB/s | 323 kB 00:00 2023-12-14T18:16:03.016 INFO:teuthology.orchestra.run.smithi093.stdout:(118/145): python3-numpy-f2py-1.14.3-10.el8.x86 1.1 MB/s | 225 kB 00:00 2023-12-14T18:16:03.032 INFO:teuthology.orchestra.run.smithi097.stdout:(96/145): openblas-0.3.15-6.el8.x86_64.rpm 16 MB/s | 5.0 MB 00:00 2023-12-14T18:16:03.052 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/161 2023-12-14T18:16:03.074 INFO:teuthology.orchestra.run.smithi097.stdout:(97/145): postgresql-contrib-10.23-2.module+el8 4.0 MB/s | 811 kB 00:00 2023-12-14T18:16:03.075 INFO:teuthology.orchestra.run.smithi093.stdout:(119/145): librdkafka-0.11.4-3.el8.x86_64.rpm 2.6 MB/s | 354 kB 00:00 2023-12-14T18:16:03.191 INFO:teuthology.orchestra.run.smithi097.stdout:(98/145): postgresql-server-10.23-2.module+el8. 19 MB/s | 5.1 MB 00:00 2023-12-14T18:16:03.192 INFO:teuthology.orchestra.run.smithi093.stdout:(120/145): openblas-threads-0.3.15-6.el8.x86_64 23 MB/s | 5.2 MB 00:00 2023-12-14T18:16:03.217 INFO:teuthology.orchestra.run.smithi097.stdout:(99/145): libpq-13.11-1.el8.x86_64.rpm 1.4 MB/s | 198 kB 00:00 2023-12-14T18:16:03.217 INFO:teuthology.orchestra.run.smithi093.stdout:(121/145): python3-mako-1.0.6-14.el8.noarch.rpm 782 kB/s | 157 kB 00:00 2023-12-14T18:16:03.250 INFO:teuthology.orchestra.run.smithi097.stdout:(100/145): automake-1.16.1-8.el8.noarch.rpm 3.2 MB/s | 714 kB 00:00 2023-12-14T18:16:03.351 INFO:teuthology.orchestra.run.smithi097.stdout:(101/145): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-12-14T18:16:03.376 INFO:teuthology.orchestra.run.smithi097.stdout:(102/145): libverto-devel-0.3.0-5.el8.x86_64.rp 143 kB/s | 18 kB 00:00 2023-12-14T18:16:03.377 INFO:teuthology.orchestra.run.smithi093.stdout:(122/145): openblas-0.3.15-6.el8.x86_64.rpm 17 MB/s | 5.0 MB 00:00 2023-12-14T18:16:03.426 INFO:teuthology.orchestra.run.smithi093.stdout:(123/145): postgresql-10.23-2.module+el8.8.0+19 6.4 MB/s | 1.5 MB 00:00 2023-12-14T18:16:03.434 INFO:teuthology.orchestra.run.smithi097.stdout:(103/145): python3-pytz-2017.2-11.el8.noarch.rp 249 kB/s | 54 kB 00:00 2023-12-14T18:16:03.459 INFO:teuthology.orchestra.run.smithi093.stdout:(124/145): postgresql-contrib-10.23-2.module+el 3.3 MB/s | 811 kB 00:00 2023-12-14T18:16:03.460 INFO:teuthology.orchestra.run.smithi097.stdout:(104/145): python3-jwt-1.6.1-2.el8.noarch.rpm 391 kB/s | 43 kB 00:00 2023-12-14T18:16:03.485 INFO:teuthology.orchestra.run.smithi097.stdout:(105/145): python3-oauthlib-2.1.0-1.el8.noarch. 1.4 MB/s | 155 kB 00:00 2023-12-14T18:16:03.568 INFO:teuthology.orchestra.run.smithi093.stdout:(125/145): automake-1.16.1-8.el8.noarch.rpm 4.9 MB/s | 714 kB 00:00 2023-12-14T18:16:03.569 INFO:teuthology.orchestra.run.smithi097.stdout:(106/145): python3-pycparser-2.14-14.el8.noarch 814 kB/s | 109 kB 00:00 2023-12-14T18:16:03.594 INFO:teuthology.orchestra.run.smithi097.stdout:(107/145): python3-requests-oauthlib-1.0.0-1.el 319 kB/s | 43 kB 00:00 2023-12-14T18:16:03.619 INFO:teuthology.orchestra.run.smithi097.stdout:(108/145): libzstd-devel-1.4.4-1.el8.x86_64.rpm 905 kB/s | 44 kB 00:00 2023-12-14T18:16:03.644 INFO:teuthology.orchestra.run.smithi097.stdout:(109/145): libaio-devel-0.3.112-1.el8.x86_64.rp 117 kB/s | 19 kB 00:00 2023-12-14T18:16:03.675 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/161 2023-12-14T18:16:03.676 INFO:teuthology.orchestra.run.smithi093.stdout:(126/145): postgresql-server-10.23-2.module+el8 17 MB/s | 5.1 MB 00:00 2023-12-14T18:16:03.702 INFO:teuthology.orchestra.run.smithi093.stdout:(127/145): libpq-13.11-1.el8.x86_64.rpm 818 kB/s | 198 kB 00:00 2023-12-14T18:16:03.727 INFO:teuthology.orchestra.run.smithi093.stdout:(128/145): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2023-12-14T18:16:03.781 INFO:teuthology.orchestra.run.smithi097.stdout:(110/145): pcre2-devel-10.32-2.el8.x86_64.rpm 3.2 MB/s | 605 kB 00:00 2023-12-14T18:16:03.804 INFO:teuthology.orchestra.run.smithi097.stdout:(111/145): pcre2-utf32-10.32-2.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2023-12-14T18:16:03.829 INFO:teuthology.orchestra.run.smithi097.stdout:(112/145): pcre2-utf16-10.32-2.el8.x86_64.rpm 1.1 MB/s | 229 kB 00:00 2023-12-14T18:16:03.867 INFO:teuthology.orchestra.run.smithi093.stdout:(129/145): python3-pytz-2017.2-11.el8.noarch.rp 284 kB/s | 54 kB 00:00 2023-12-14T18:16:03.868 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/161 2023-12-14T18:16:03.904 INFO:teuthology.orchestra.run.smithi097.stdout:(113/145): libselinux-devel-2.9-5.el8.x86_64.rp 1.6 MB/s | 200 kB 00:00 2023-12-14T18:16:03.929 INFO:teuthology.orchestra.run.smithi097.stdout:(114/145): keyutils-libs-devel-1.5.10-9.el8.x86 382 kB/s | 48 kB 00:00 2023-12-14T18:16:03.955 INFO:teuthology.orchestra.run.smithi097.stdout:(115/145): libsepol-devel-2.9-3.el8.x86_64.rpm 692 kB/s | 87 kB 00:00 2023-12-14T18:16:04.002 INFO:teuthology.orchestra.run.smithi093.stdout:(130/145): openssl-1.1.1k-9.el8_7.x86_64.rpm 5.2 MB/s | 710 kB 00:00 2023-12-14T18:16:04.055 INFO:teuthology.orchestra.run.smithi097.stdout:(116/145): zlib-devel-1.2.11-18.el8_5.x86_64.rp 382 kB/s | 58 kB 00:00 2023-12-14T18:16:04.080 INFO:teuthology.orchestra.run.smithi097.stdout:(117/145): libcom_err-devel-1.45.6-4.el8.x86_64 256 kB/s | 39 kB 00:00 2023-12-14T18:16:04.144 INFO:teuthology.orchestra.run.smithi093.stdout:(131/145): librbd1-18.0.0-3905.g1faa3533.el8.x8 8.5 MB/s | 3.6 MB 00:00 2023-12-14T18:16:04.147 INFO:teuthology.orchestra.run.smithi097.stdout:(118/145): openssl-devel-1.1.1k-9.el8_7.x86_64. 12 MB/s | 2.3 MB 00:00 2023-12-14T18:16:04.173 INFO:teuthology.orchestra.run.smithi097.stdout:(119/145): fuse-2.9.7-17.el8.x86_64.rpm 708 kB/s | 83 kB 00:00 2023-12-14T18:16:04.203 INFO:teuthology.orchestra.run.smithi093.stdout:(132/145): openssl-libs-1.1.1k-9.el8_7.x86_64.r 7.3 MB/s | 1.5 MB 00:00 2023-12-14T18:16:04.206 INFO:teuthology.orchestra.run.smithi097.stdout:(120/145): libgfortran-8.5.0-20.el8.x86_64.rpm 5.0 MB/s | 645 kB 00:00 2023-12-14T18:16:04.253 INFO:teuthology.orchestra.run.smithi093.stdout:(133/145): elfutils-libelf-0.189-3.el8.x86_64.r 2.1 MB/s | 232 kB 00:00 2023-12-14T18:16:04.298 INFO:teuthology.orchestra.run.smithi097.stdout:(121/145): libuuid-devel-2.32.1-43.el8.x86_64.r 797 kB/s | 100 kB 00:00 2023-12-14T18:16:04.312 INFO:teuthology.orchestra.run.smithi093.stdout:(134/145): librados2-18.0.0-3905.g1faa3533.el8. 6.0 MB/s | 3.7 MB 00:00 2023-12-14T18:16:04.332 INFO:teuthology.orchestra.run.smithi097.stdout:(122/145): libquadmath-8.5.0-20.el8.x86_64.rpm 936 kB/s | 172 kB 00:00 2023-12-14T18:16:04.357 INFO:teuthology.orchestra.run.smithi097.stdout:(123/145): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.5 MB/s | 238 kB 00:00 2023-12-14T18:16:04.371 INFO:teuthology.orchestra.run.smithi093.stdout:(135/145): elfutils-libs-0.189-3.el8.x86_64.rpm 1.8 MB/s | 303 kB 00:00 2023-12-14T18:16:04.396 INFO:teuthology.orchestra.run.smithi093.stdout:(136/145): libmount-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 237 kB 00:00 2023-12-14T18:16:04.438 INFO:teuthology.orchestra.run.smithi093.stdout:(137/145): libsmartcols-2.32.1-43.el8.x86_64.rp 1.4 MB/s | 179 kB 00:00 2023-12-14T18:16:04.441 INFO:teuthology.orchestra.run.smithi097.stdout:(124/145): NetworkManager-initscripts-updown-1. 1.7 MB/s | 143 kB 00:00 2023-12-14T18:16:04.474 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/161 2023-12-14T18:16:04.483 INFO:teuthology.orchestra.run.smithi097.stdout:(125/145): python3-cryptography-3.2.1-6.el8.x86 3.0 MB/s | 559 kB 00:00 2023-12-14T18:16:04.521 INFO:teuthology.orchestra.run.smithi093.stdout:(138/145): libuuid-2.32.1-43.el8.x86_64.rpm 657 kB/s | 99 kB 00:00 2023-12-14T18:16:04.547 INFO:teuthology.orchestra.run.smithi093.stdout:(139/145): util-linux-user-2.32.1-43.el8.x86_64 946 kB/s | 103 kB 00:00 2023-12-14T18:16:04.591 INFO:teuthology.orchestra.run.smithi097.stdout:(126/145): elfutils-libelf-devel-0.189-3.el8.x8 410 kB/s | 62 kB 00:00 2023-12-14T18:16:04.613 INFO:teuthology.orchestra.run.smithi093.stdout:(140/145): xfsdump-3.1.8-6.el8.x86_64.rpm 3.5 MB/s | 333 kB 00:00 2023-12-14T18:16:04.617 INFO:teuthology.orchestra.run.smithi097.stdout:(127/145): xfsprogs-devel-5.0.0-12.el8.x86_64.r 255 kB/s | 73 kB 00:00 2023-12-14T18:16:04.642 INFO:teuthology.orchestra.run.smithi097.stdout:(128/145): librabbitmq-0.9.0-4.el8.x86_64.rpm 296 kB/s | 47 kB 00:00 2023-12-14T18:16:04.677 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/161 2023-12-14T18:16:04.692 INFO:teuthology.orchestra.run.smithi097.stdout:(129/145): krb5-devel-1.18.2-26.el8_9.x86_64.rp 5.5 MB/s | 562 kB 00:00 2023-12-14T18:16:04.714 INFO:teuthology.orchestra.run.smithi093.stdout:(141/145): libblkid-2.32.1-43.el8.x86_64.rpm 2.2 MB/s | 220 kB 00:00 2023-12-14T18:16:04.764 INFO:teuthology.orchestra.run.smithi093.stdout:(142/145): iproute-6.2.0-5.el8_9.x86_64.rpm 3.8 MB/s | 854 kB 00:00 2023-12-14T18:16:04.841 INFO:teuthology.orchestra.run.smithi093.stdout:(143/145): util-linux-2.32.1-43.el8.x86_64.rpm 5.6 MB/s | 2.5 MB 00:00 2023-12-14T18:16:04.870 INFO:teuthology.orchestra.run.smithi097.stdout:(130/145): openssl-1.1.1k-9.el8_7.x86_64.rpm 3.9 MB/s | 710 kB 00:00 2023-12-14T18:16:04.873 INFO:teuthology.orchestra.run.smithi093.stdout:(144/145): libfdisk-2.32.1-43.el8.x86_64.rpm 1.6 MB/s | 254 kB 00:00 2023-12-14T18:16:04.956 INFO:teuthology.orchestra.run.smithi093.stdout:(145/145): xfsprogs-5.0.0-12.el8.x86_64.rpm 5.6 MB/s | 1.1 MB 00:00 2023-12-14T18:16:04.959 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:16:04.959 INFO:teuthology.orchestra.run.smithi093.stdout:Total 13 MB/s | 205 MB 00:16 2023-12-14T18:16:05.009 INFO:teuthology.orchestra.run.smithi097.stdout:(131/145): librbd1-18.0.0-3905.g1faa3533.el8.x8 9.7 MB/s | 3.6 MB 00:00 2023-12-14T18:16:05.051 INFO:teuthology.orchestra.run.smithi097.stdout:(132/145): openssl-libs-1.1.1k-9.el8_7.x86_64.r 8.1 MB/s | 1.5 MB 00:00 2023-12-14T18:16:05.093 INFO:teuthology.orchestra.run.smithi097.stdout:(133/145): elfutils-libelf-0.189-3.el8.x86_64.r 2.7 MB/s | 232 kB 00:00 2023-12-14T18:16:05.193 INFO:teuthology.orchestra.run.smithi097.stdout:(134/145): elfutils-libs-0.189-3.el8.x86_64.rpm 2.1 MB/s | 303 kB 00:00 2023-12-14T18:16:05.219 INFO:teuthology.orchestra.run.smithi097.stdout:(135/145): libmount-2.32.1-43.el8.x86_64.rpm 1.8 MB/s | 237 kB 00:00 2023-12-14T18:16:05.327 INFO:teuthology.orchestra.run.smithi097.stdout:(136/145): libsmartcols-2.32.1-43.el8.x86_64.rp 1.3 MB/s | 179 kB 00:00 2023-12-14T18:16:05.353 INFO:teuthology.orchestra.run.smithi097.stdout:(137/145): libuuid-2.32.1-43.el8.x86_64.rpm 741 kB/s | 99 kB 00:00 2023-12-14T18:16:05.381 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/161 2023-12-14T18:16:05.478 INFO:teuthology.orchestra.run.smithi097.stdout:(138/145): util-linux-2.32.1-43.el8.x86_64.rpm 16 MB/s | 2.5 MB 00:00 2023-12-14T18:16:05.504 INFO:teuthology.orchestra.run.smithi097.stdout:(139/145): util-linux-user-2.32.1-43.el8.x86_64 682 kB/s | 103 kB 00:00 2023-12-14T18:16:05.629 INFO:teuthology.orchestra.run.smithi097.stdout:(140/145): xfsdump-3.1.8-6.el8.x86_64.rpm 2.2 MB/s | 333 kB 00:00 2023-12-14T18:16:05.649 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/161 2023-12-14T18:16:05.662 INFO:teuthology.orchestra.run.smithi097.stdout:(141/145): iproute-6.2.0-5.el8_9.x86_64.rpm 5.3 MB/s | 854 kB 00:00 2023-12-14T18:16:05.779 INFO:teuthology.orchestra.run.smithi097.stdout:(142/145): libblkid-2.32.1-43.el8.x86_64.rpm 1.4 MB/s | 220 kB 00:00 2023-12-14T18:16:05.807 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/161 2023-12-14T18:16:05.872 INFO:teuthology.orchestra.run.smithi097.stdout:(143/145): libfdisk-2.32.1-43.el8.x86_64.rpm 1.2 MB/s | 254 kB 00:00 2023-12-14T18:16:05.913 INFO:teuthology.orchestra.run.smithi097.stdout:(144/145): xfsprogs-5.0.0-12.el8.x86_64.rpm 8.0 MB/s | 1.1 MB 00:00 2023-12-14T18:16:05.971 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/161 2023-12-14T18:16:05.987 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction check 2023-12-14T18:16:06.088 INFO:teuthology.orchestra.run.smithi097.stdout:(145/145): librados2-18.0.0-3905.g1faa3533.el8. 2.5 MB/s | 3.7 MB 00:01 2023-12-14T18:16:06.092 INFO:teuthology.orchestra.run.smithi097.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:16:06.092 INFO:teuthology.orchestra.run.smithi097.stdout:Total 12 MB/s | 205 MB 00:16 2023-12-14T18:16:06.269 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/161 2023-12-14T18:16:06.379 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction check succeeded. 2023-12-14T18:16:06.380 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction test 2023-12-14T18:16:06.540 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/161 2023-12-14T18:16:06.793 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/161 2023-12-14T18:16:06.966 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 31/161 2023-12-14T18:16:07.104 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 32/161 2023-12-14T18:16:07.157 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction check 2023-12-14T18:16:07.354 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 33/161 2023-12-14T18:16:07.556 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction check succeeded. 2023-12-14T18:16:07.556 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction test 2023-12-14T18:16:07.907 INFO:teuthology.orchestra.run.smithi093.stdout:Transaction test succeeded. 2023-12-14T18:16:07.917 INFO:teuthology.orchestra.run.smithi093.stdout:Running transaction 2023-12-14T18:16:08.338 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/161 2023-12-14T18:16:08.656 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/161 2023-12-14T18:16:08.912 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/161 2023-12-14T18:16:09.078 INFO:teuthology.orchestra.run.smithi097.stdout:Transaction test succeeded. 2023-12-14T18:16:09.089 INFO:teuthology.orchestra.run.smithi097.stdout:Running transaction 2023-12-14T18:16:09.101 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/161 2023-12-14T18:16:10.869 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/161 2023-12-14T18:16:11.193 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 38/161 2023-12-14T18:16:11.404 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2023-12-14T18:16:11.644 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/161 2023-12-14T18:16:11.870 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 41/161 2023-12-14T18:16:12.074 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 42/161 2023-12-14T18:16:12.292 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:16:12.376 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:16:12.600 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:16:12.667 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:16:12.907 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/161 2023-12-14T18:16:13.155 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/161 2023-12-14T18:16:13.342 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:16:13.856 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:16:14.050 INFO:teuthology.orchestra.run.smithi093.stdout: Preparing : 1/1 2023-12-14T18:16:14.259 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:16:14.261 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/1 2023-12-14T18:16:14.277 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:16:14.463 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/161 2023-12-14T18:16:14.541 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:14.698 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/161 2023-12-14T18:16:14.789 INFO:teuthology.orchestra.run.smithi097.stdout: Preparing : 1/1 2023-12-14T18:16:14.790 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:14.897 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/161 2023-12-14T18:16:15.000 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/161 2023-12-14T18:16:15.002 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/1 2023-12-14T18:16:15.087 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/161 2023-12-14T18:16:15.205 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:15.216 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:15.287 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/161 2023-12-14T18:16:15.293 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:15.477 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:1.1.1k-9.el8_7.x86_64 1/161 2023-12-14T18:16:15.505 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:15.596 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:15.612 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/161 2023-12-14T18:16:15.713 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : thrift-0.13.0-2.el8.x86_64 2/161 2023-12-14T18:16:15.797 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/161 2023-12-14T18:16:15.897 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/161 2023-12-14T18:16:15.922 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:16.009 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 3/161 2023-12-14T18:16:16.038 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/161 2023-12-14T18:16:16.146 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/161 2023-12-14T18:16:16.214 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:16.287 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 4/161 2023-12-14T18:16:16.321 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/161 2023-12-14T18:16:16.446 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 7/161 2023-12-14T18:16:16.522 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : liboath-2.6.2-3.el8.x86_64 5/161 2023-12-14T18:16:16.531 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 7/161 2023-12-14T18:16:16.587 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/161 2023-12-14T18:16:16.763 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:16.947 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 6/161 2023-12-14T18:16:16.971 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/161 2023-12-14T18:16:17.020 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 6/161 2023-12-14T18:16:17.170 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:17.262 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 7/161 2023-12-14T18:16:17.286 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 60/161 2023-12-14T18:16:17.405 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:17.476 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/161 2023-12-14T18:16:17.488 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:17.706 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 62/161 2023-12-14T18:16:17.801 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:17.882 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 8/161 2023-12-14T18:16:17.912 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 63/161 2023-12-14T18:16:17.996 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:18.122 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:18.151 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:18.282 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 64/161 2023-12-14T18:16:18.354 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:16:18.449 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:16:18.482 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 65/161 2023-12-14T18:16:18.503 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:16:18.687 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 66/161 2023-12-14T18:16:18.705 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:18.791 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:16:18.836 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:16:18.927 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/161 2023-12-14T18:16:19.055 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:16:19.104 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:19.123 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/161 2023-12-14T18:16:19.158 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:16:19.192 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:19.431 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libpq-13.11-1.el8.x86_64 14/161 2023-12-14T18:16:19.443 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/161 2023-12-14T18:16:19.527 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:16:19.645 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : automake-1.16.1-8.el8.noarch 70/161 2023-12-14T18:16:19.715 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/161 2023-12-14T18:16:19.830 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:19.849 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : uuid-1.6.2-43.el8.x86_64 71/161 2023-12-14T18:16:19.917 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 13/161 2023-12-14T18:16:19.924 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2023-12-14T18:16:19.928 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 71/161 2023-12-14T18:16:20.133 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 72/161 2023-12-14T18:16:20.138 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libpq-13.11-1.el8.x86_64 14/161 2023-12-14T18:16:20.211 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 72/161 2023-12-14T18:16:20.320 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/161 2023-12-14T18:16:20.368 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 15/161 2023-12-14T18:16:20.416 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : jq-1.6-7.el8.x86_64 73/161 2023-12-14T18:16:20.542 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/161 2023-12-14T18:16:20.566 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 18/161 2023-12-14T18:16:20.615 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/161 2023-12-14T18:16:20.721 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:16:20.815 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/161 2023-12-14T18:16:20.873 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:16:20.948 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 17/161 2023-12-14T18:16:20.993 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/161 2023-12-14T18:16:21.063 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:16:21.138 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:16:21.158 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 77/161 2023-12-14T18:16:21.192 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 18/161 2023-12-14T18:16:21.330 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 78/161 2023-12-14T18:16:21.332 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 21/161 2023-12-14T18:16:21.346 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:16:21.449 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:16:21.630 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 22/161 2023-12-14T18:16:21.631 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:16:21.646 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 79/161 2023-12-14T18:16:21.708 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 20/161 2023-12-14T18:16:21.847 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 80/161 2023-12-14T18:16:21.888 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/161 2023-12-14T18:16:21.888 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 23/161 2023-12-14T18:16:22.007 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/161 2023-12-14T18:16:22.062 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 81/161 2023-12-14T18:16:22.104 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 24/161 2023-12-14T18:16:22.196 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/161 2023-12-14T18:16:22.236 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 24/161 2023-12-14T18:16:22.252 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 82/161 2023-12-14T18:16:22.412 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 25/161 2023-12-14T18:16:22.433 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 83/161 2023-12-14T18:16:22.693 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 84/161 2023-12-14T18:16:22.805 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/161 2023-12-14T18:16:22.927 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 85/161 2023-12-14T18:16:22.971 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 23/161 2023-12-14T18:16:23.040 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 25/161 2023-12-14T18:16:23.135 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 86/161 2023-12-14T18:16:23.179 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 26/161 2023-12-14T18:16:23.297 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 87/161 2023-12-14T18:16:23.544 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 88/161 2023-12-14T18:16:23.595 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 23/161 2023-12-14T18:16:23.746 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openblas-0.3.15-6.el8.x86_64 24/161 2023-12-14T18:16:23.769 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 89/161 2023-12-14T18:16:23.814 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 26/161 2023-12-14T18:16:23.971 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 90/161 2023-12-14T18:16:24.012 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openblas-0.3.15-6.el8.x86_64 27/161 2023-12-14T18:16:24.219 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-portend-2.6-1.el8.noarch 91/161 2023-12-14T18:16:24.434 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 24/161 2023-12-14T18:16:24.465 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 92/161 2023-12-14T18:16:24.673 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 93/161 2023-12-14T18:16:24.702 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/161 2023-12-14T18:16:24.702 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 27/161 2023-12-14T18:16:24.885 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 26/161 2023-12-14T18:16:24.906 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 94/161 2023-12-14T18:16:24.984 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 28/161 2023-12-14T18:16:25.033 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/161 2023-12-14T18:16:25.176 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : zlib-devel-1.2.11-18.el8_5.x86_64 29/161 2023-12-14T18:16:25.323 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/161 2023-12-14T18:16:25.429 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/161 2023-12-14T18:16:25.593 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 31/161 2023-12-14T18:16:25.609 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 29/161 2023-12-14T18:16:25.756 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 32/161 2023-12-14T18:16:25.871 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 30/161 2023-12-14T18:16:26.011 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 95/161 2023-12-14T18:16:26.033 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 33/161 2023-12-14T18:16:26.052 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 31/161 2023-12-14T18:16:26.165 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 32/161 2023-12-14T18:16:26.245 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2023-12-14T18:16:26.344 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 96/161 2023-12-14T18:16:26.439 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 33/161 2023-12-14T18:16:26.572 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2023-12-14T18:16:26.873 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 98/161 2023-12-14T18:16:27.441 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 34/161 2023-12-14T18:16:27.617 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 99/161 2023-12-14T18:16:27.751 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 35/161 2023-12-14T18:16:27.863 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/161 2023-12-14T18:16:28.005 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 34/161 2023-12-14T18:16:28.007 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 36/161 2023-12-14T18:16:28.212 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/161 2023-12-14T18:16:28.372 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 35/161 2023-12-14T18:16:28.671 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/161 2023-12-14T18:16:28.865 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:16:28.971 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:16:29.198 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 103/161 2023-12-14T18:16:29.392 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 36/161 2023-12-14T18:16:29.758 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 37/161 2023-12-14T18:16:29.973 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 37/161 2023-12-14T18:16:30.021 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 38/161 2023-12-14T18:16:30.264 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2023-12-14T18:16:30.330 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 38/161 2023-12-14T18:16:30.513 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/161 2023-12-14T18:16:30.598 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rsa-4.9-2.el8.noarch 39/161 2023-12-14T18:16:30.731 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 41/161 2023-12-14T18:16:30.840 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 40/161 2023-12-14T18:16:30.932 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 42/161 2023-12-14T18:16:31.040 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 41/161 2023-12-14T18:16:31.127 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:16:31.187 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:16:31.215 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 42/161 2023-12-14T18:16:31.412 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:16:31.436 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:16:31.491 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 43/161 2023-12-14T18:16:31.496 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:16:31.695 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:16:31.766 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/161 2023-12-14T18:16:32.000 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/161 2023-12-14T18:16:32.150 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 45/161 2023-12-14T18:16:32.173 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/161 2023-12-14T18:16:32.378 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:16:32.401 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 46/161 2023-12-14T18:16:32.610 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:16:33.017 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:16:33.031 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:16:33.119 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/161 2023-12-14T18:16:33.437 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:16:33.454 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:16:33.486 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:16:33.498 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:16:33.579 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/161 2023-12-14T18:16:33.649 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : fuse-2.9.7-17.el8.x86_64 49/161 2023-12-14T18:16:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : automake-1.16.1-8.el8.noarch 49/161 2023-12-14T18:16:33.874 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 50/161 2023-12-14T18:16:34.032 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 51/161 2023-12-14T18:16:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : uuid-1.6.2-43.el8.x86_64 50/161 2023-12-14T18:16:34.095 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:16:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 50/161 2023-12-14T18:16:34.231 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 52/161 2023-12-14T18:16:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 51/161 2023-12-14T18:16:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 51/161 2023-12-14T18:16:34.401 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:16:34.439 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 53/161 2023-12-14T18:16:34.487 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:16:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : jq-1.6-7.el8.x86_64 52/161 2023-12-14T18:16:34.622 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:16:34.703 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 54/161 2023-12-14T18:16:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 53/161 2023-12-14T18:16:34.922 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 55/161 2023-12-14T18:16:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 54/161 2023-12-14T18:16:35.133 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 56/161 2023-12-14T18:16:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 55/161 2023-12-14T18:16:35.273 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 57/161 2023-12-14T18:16:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : fuse-2.9.7-17.el8.x86_64 56/161 2023-12-14T18:16:35.514 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 58/161 2023-12-14T18:16:35.659 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcom_err-devel-1.45.6-4.el8.x86_64 57/161 2023-12-14T18:16:35.865 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 58/161 2023-12-14T18:16:35.892 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 59/161 2023-12-14T18:16:36.096 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 59/161 2023-12-14T18:16:36.197 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 60/161 2023-12-14T18:16:36.322 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf32-10.32-2.el8.x86_64 60/161 2023-12-14T18:16:36.370 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 61/161 2023-12-14T18:16:36.597 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-utf16-10.32-2.el8.x86_64 61/161 2023-12-14T18:16:36.651 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 62/161 2023-12-14T18:16:36.898 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : pcre2-devel-10.32-2.el8.x86_64 62/161 2023-12-14T18:16:36.932 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 63/161 2023-12-14T18:16:37.148 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libselinux-devel-2.9-5.el8.x86_64 63/161 2023-12-14T18:16:37.269 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 64/161 2023-12-14T18:16:37.373 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 64/161 2023-12-14T18:16:37.494 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 65/161 2023-12-14T18:16:37.562 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 65/161 2023-12-14T18:16:37.720 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 66/161 2023-12-14T18:16:37.916 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 66/161 2023-12-14T18:16:37.921 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 67/161 2023-12-14T18:16:38.191 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 68/161 2023-12-14T18:16:38.236 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 67/161 2023-12-14T18:16:38.443 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 68/161 2023-12-14T18:16:38.512 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 69/161 2023-12-14T18:16:38.697 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 69/161 2023-12-14T18:16:38.737 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : automake-1.16.1-8.el8.noarch 70/161 2023-12-14T18:16:38.936 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : uuid-1.6.2-43.el8.x86_64 71/161 2023-12-14T18:16:38.964 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 70/161 2023-12-14T18:16:39.004 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: uuid-1.6.2-43.el8.x86_64 71/161 2023-12-14T18:16:39.202 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 72/161 2023-12-14T18:16:39.269 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 72/161 2023-12-14T18:16:39.292 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 71/161 2023-12-14T18:16:39.494 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : jq-1.6-7.el8.x86_64 73/161 2023-12-14T18:16:39.542 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 72/161 2023-12-14T18:16:39.684 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 74/161 2023-12-14T18:16:39.793 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 73/161 2023-12-14T18:16:39.894 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 75/161 2023-12-14T18:16:40.037 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 74/161 2023-12-14T18:16:40.096 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 76/161 2023-12-14T18:16:40.244 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 77/161 2023-12-14T18:16:40.269 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libverto-devel-0.3.0-5.el8.x86_64 75/161 2023-12-14T18:16:40.440 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 78/161 2023-12-14T18:16:40.545 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : krb5-devel-1.18.2-26.el8_9.x86_64 76/161 2023-12-14T18:16:40.735 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 77/161 2023-12-14T18:16:40.740 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 79/161 2023-12-14T18:16:40.947 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 78/161 2023-12-14T18:16:40.984 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 80/161 2023-12-14T18:16:41.148 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 81/161 2023-12-14T18:16:41.222 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 79/161 2023-12-14T18:16:41.338 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 82/161 2023-12-14T18:16:41.474 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 80/161 2023-12-14T18:16:41.520 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 83/161 2023-12-14T18:16:41.689 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 81/161 2023-12-14T18:16:41.745 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 84/161 2023-12-14T18:16:41.879 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 82/161 2023-12-14T18:16:41.972 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 85/161 2023-12-14T18:16:42.094 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 83/161 2023-12-14T18:16:42.189 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 86/161 2023-12-14T18:16:42.341 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 87/161 2023-12-14T18:16:42.351 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 84/161 2023-12-14T18:16:42.572 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 88/161 2023-12-14T18:16:42.595 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 85/161 2023-12-14T18:16:42.781 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 89/161 2023-12-14T18:16:42.813 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 86/161 2023-12-14T18:16:42.941 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 90/161 2023-12-14T18:16:42.965 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 87/161 2023-12-14T18:16:43.188 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-portend-2.6-1.el8.noarch 91/161 2023-12-14T18:16:43.229 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 88/161 2023-12-14T18:16:43.442 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 92/161 2023-12-14T18:16:43.454 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 89/161 2023-12-14T18:16:43.609 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 93/161 2023-12-14T18:16:43.648 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 90/161 2023-12-14T18:16:43.859 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 94/161 2023-12-14T18:16:43.920 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-portend-2.6-1.el8.noarch 91/161 2023-12-14T18:16:44.175 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 92/161 2023-12-14T18:16:44.383 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 93/161 2023-12-14T18:16:44.616 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 94/161 2023-12-14T18:16:44.986 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 95/161 2023-12-14T18:16:45.322 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 96/161 2023-12-14T18:16:45.551 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2023-12-14T18:16:45.713 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 95/161 2023-12-14T18:16:45.885 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 98/161 2023-12-14T18:16:46.037 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 96/161 2023-12-14T18:16:46.165 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 99/161 2023-12-14T18:16:46.274 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/161 2023-12-14T18:16:46.360 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/161 2023-12-14T18:16:46.609 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 98/161 2023-12-14T18:16:46.921 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 99/161 2023-12-14T18:16:47.146 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/161 2023-12-14T18:16:47.150 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 100/161 2023-12-14T18:16:47.351 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:16:47.432 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:16:47.610 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 103/161 2023-12-14T18:16:47.921 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 101/161 2023-12-14T18:16:48.125 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:16:48.238 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 102/161 2023-12-14T18:16:48.467 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 103/161 2023-12-14T18:16:51.586 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:16:51.860 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:16:52.275 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:16:52.457 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:16:52.567 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:16:52.753 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:16:52.831 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:16:53.042 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:16:53.176 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 104/161 2023-12-14T18:16:53.925 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:16:53.925 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /sys 2023-12-14T18:16:53.925 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /proc 2023-12-14T18:16:53.925 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /dev 2023-12-14T18:16:53.925 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /run 2023-12-14T18:16:53.926 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /mnt 2023-12-14T18:16:53.926 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /var/tmp 2023-12-14T18:16:53.926 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /home 2023-12-14T18:16:53.926 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /tmp 2023-12-14T18:16:53.926 INFO:teuthology.orchestra.run.smithi160.stdout:skipping the directory /dev 2023-12-14T18:16:53.926 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:16:54.253 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:16:54.269 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:16:54.350 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 105/161 2023-12-14T18:16:54.524 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:16:55.846 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:16:56.055 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:16:56.275 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:16:56.471 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:16:56.929 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:16:57.077 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:16:57.095 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:16:58.825 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:16:59.008 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:16:59.141 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 112/161 2023-12-14T18:16:59.350 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 113/161 2023-12-14T18:16:59.577 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:16:59.794 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:17:00.129 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:17:01.095 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:17:01.284 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:17:01.377 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:17:01.741 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 117/161 2023-12-14T18:17:01.913 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:17:01.949 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:17:02.118 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:17:02.130 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:17:03.397 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 120/161 2023-12-14T18:17:03.638 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:17:03.672 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:17:03.876 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:17:03.919 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:17:04.134 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:17:04.481 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:17:04.684 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:17:08.595 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:17:09.049 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 125/161 2023-12-14T18:17:09.438 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 126/161 2023-12-14T18:17:09.675 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libtool-2.4.6-25.el8.x86_64 127/161 2023-12-14T18:17:10.371 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 127/161 2023-12-14T18:17:10.679 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : openssl-devel-1:1.1.1k-9.el8_7.x86_64 128/161 2023-12-14T18:17:10.973 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 129/161 2023-12-14T18:17:11.181 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 130/161 2023-12-14T18:17:11.419 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 131/161 2023-12-14T18:17:11.629 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 132/161 2023-12-14T18:17:11.793 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 133/161 2023-12-14T18:17:12.329 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /sys 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /proc 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /run 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /mnt 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /var/tmp 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /home 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /tmp 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout:skipping the directory /dev 2023-12-14T18:17:12.330 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:17:12.715 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:17:13.703 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 106/161 2023-12-14T18:17:13.703 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /sys 2023-12-14T18:17:13.704 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /proc 2023-12-14T18:17:13.704 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2023-12-14T18:17:13.704 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /run 2023-12-14T18:17:13.704 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /mnt 2023-12-14T18:17:13.704 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /var/tmp 2023-12-14T18:17:13.704 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /home 2023-12-14T18:17:13.704 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /tmp 2023-12-14T18:17:13.704 INFO:teuthology.orchestra.run.smithi093.stdout:skipping the directory /dev 2023-12-14T18:17:13.704 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:17:14.039 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:14.113 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:17:14.302 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:17:14.341 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:14.389 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:14.492 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:17:14.711 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:17:14.877 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 135/161 2023-12-14T18:17:14.886 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:17:15.260 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 136/161 2023-12-14T18:17:15.345 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:17:15.540 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:17:15.553 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 137/161 2023-12-14T18:17:15.557 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:17:15.692 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 107/161 2023-12-14T18:17:15.749 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 138/161 2023-12-14T18:17:15.899 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:17:15.982 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 139/161 2023-12-14T18:17:16.118 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 108/161 2023-12-14T18:17:16.246 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:17:16.297 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 140/161 2023-12-14T18:17:16.576 INFO:teuthology.orchestra.run.smithi160.stdout: Upgrading : openssl-1:1.1.1k-9.el8_7.x86_64 141/161 2023-12-14T18:17:16.701 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 109/161 2023-12-14T18:17:16.782 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 142/161 2023-12-14T18:17:16.792 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 142/161 2023-12-14T18:17:16.896 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:17:16.914 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 110/161 2023-12-14T18:17:17.034 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 143/161 2023-12-14T18:17:17.265 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : flex-2.6.1-9.el8.x86_64 144/161 2023-12-14T18:17:17.338 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:17:17.354 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 144/161 2023-12-14T18:17:17.503 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:17:17.599 INFO:teuthology.orchestra.run.smithi160.stdout: Installing : bison-3.0.4-10.el8.x86_64 145/161 2023-12-14T18:17:17.613 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 145/161 2023-12-14T18:17:17.654 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 112/161 2023-12-14T18:17:17.812 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 113/161 2023-12-14T18:17:17.823 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 146/161 2023-12-14T18:17:18.001 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 147/161 2023-12-14T18:17:18.056 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:17:18.211 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 148/161 2023-12-14T18:17:18.212 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:17:18.276 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:17:18.502 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:17:18.502 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:17:18.583 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:17:18.744 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:17:18.744 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:17:18.855 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:17:19.021 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:17:19.059 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 111/161 2023-12-14T18:17:19.193 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 112/161 2023-12-14T18:17:19.258 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 152/161 2023-12-14T18:17:19.411 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 113/161 2023-12-14T18:17:19.474 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 153/161 2023-12-14T18:17:19.548 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:17:19.662 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:17:19.717 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 154/161 2023-12-14T18:17:19.717 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:17:19.755 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:17:19.849 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:17:19.875 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 114/161 2023-12-14T18:17:19.993 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:17:19.993 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:17:20.123 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:17:20.244 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 117/161 2023-12-14T18:17:20.301 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:17:20.301 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:17:20.399 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:17:20.437 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:17:20.592 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:17:20.593 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:17:20.605 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:17:20.617 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:17:20.842 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:17:20.842 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:17:21.085 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 115/161 2023-12-14T18:17:21.100 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:17:21.279 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:17:21.292 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 160/161 2023-12-14T18:17:21.292 INFO:teuthology.orchestra.run.smithi160.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:17:21.371 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 116/161 2023-12-14T18:17:21.759 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 117/161 2023-12-14T18:17:21.863 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 120/161 2023-12-14T18:17:21.965 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:17:22.001 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 118/161 2023-12-14T18:17:22.132 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:17:22.168 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:17:22.170 INFO:teuthology.orchestra.run.smithi160.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 1/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 3/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 4/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 5/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 6/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 7/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 8/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:17:22.171 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 13/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 14/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 15/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 16/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 17/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 18/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 20/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 21/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 22/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 23/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 24/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 25/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 26/161 2023-12-14T18:17:22.172 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 27/161 2023-12-14T18:17:22.173 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 28/161 2023-12-14T18:17:22.173 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 29/161 2023-12-14T18:17:22.173 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 30/161 2023-12-14T18:17:22.182 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:17:22.190 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 31/161 2023-12-14T18:17:22.190 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 32/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 33/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 34/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 35/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 36/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 37/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 38/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/161 2023-12-14T18:17:22.191 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/161 2023-12-14T18:17:22.192 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/161 2023-12-14T18:17:22.192 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/161 2023-12-14T18:17:22.192 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/161 2023-12-14T18:17:22.192 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/161 2023-12-14T18:17:22.192 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/161 2023-12-14T18:17:22.192 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/161 2023-12-14T18:17:22.192 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/161 2023-12-14T18:17:22.192 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/161 2023-12-14T18:17:22.192 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/161 2023-12-14T18:17:22.193 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/161 2023-12-14T18:17:22.240 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/161 2023-12-14T18:17:22.240 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/161 2023-12-14T18:17:22.240 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/161 2023-12-14T18:17:22.240 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/161 2023-12-14T18:17:22.240 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/161 2023-12-14T18:17:22.240 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/161 2023-12-14T18:17:22.240 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/161 2023-12-14T18:17:22.241 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 83/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 86/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 87/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 88/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 89/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 90/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 91/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 92/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 93/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 94/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 95/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 96/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 97/161 2023-12-14T18:17:22.242 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : postgresql-server-10.23-2.module+el8.8.0+19493+5 98/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : automake-1.16.1-8.el8.noarch 99/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libpq-13.11-1.el8.x86_64 100/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : jq-1.6-7.el8.x86_64 101/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 102/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 103/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 104/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 105/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 106/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 107/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 108/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 109/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 110/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 112/161 2023-12-14T18:17:22.243 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 113/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 114/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 115/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 117/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8_7.x86_64 118/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 124/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 125/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 126/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 127/161 2023-12-14T18:17:22.244 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 128/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 129/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 130/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 132/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:1.1.1k-9.el8_7.x86_64 134/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 135/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8_7.x86_64 136/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 138/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 139/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 140/161 2023-12-14T18:17:22.245 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 141/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 143/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 145/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 147/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 149/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 151/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 154/161 2023-12-14T18:17:22.246 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 155/161 2023-12-14T18:17:22.247 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 156/161 2023-12-14T18:17:22.247 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:17:22.247 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 158/161 2023-12-14T18:17:22.247 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:17:22.247 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2023-12-14T18:17:22.247 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 119/161 2023-12-14T18:17:22.364 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:17:22.407 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:17:22.564 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:17:22.911 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:17:23.097 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:17:23.452 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 120/161 2023-12-14T18:17:23.690 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:17:23.724 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 121/161 2023-12-14T18:17:23.937 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:17:23.979 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 122/161 2023-12-14T18:17:24.178 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:17:24.525 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 123/161 2023-12-14T18:17:24.729 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:17:26.999 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:17:27.450 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 125/161 2023-12-14T18:17:27.831 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 126/161 2023-12-14T18:17:28.054 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libtool-2.4.6-25.el8.x86_64 127/161 2023-12-14T18:17:28.634 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 124/161 2023-12-14T18:17:28.743 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 127/161 2023-12-14T18:17:29.059 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : openssl-devel-1:1.1.1k-9.el8_7.x86_64 128/161 2023-12-14T18:17:29.360 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 129/161 2023-12-14T18:17:29.556 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 125/161 2023-12-14T18:17:29.619 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 130/161 2023-12-14T18:17:29.857 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : openssl-devel-1:1.1.1k-9.el8_7.x86_64 126/161 2023-12-14T18:17:29.874 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 131/161 2023-12-14T18:17:30.109 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 132/161 2023-12-14T18:17:30.158 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/161 2023-12-14T18:17:30.272 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 133/161 2023-12-14T18:17:30.621 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 128/161 2023-12-14T18:17:31.009 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 129/161 2023-12-14T18:17:31.244 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libtool-2.4.6-25.el8.x86_64 130/161 2023-12-14T18:17:31.257 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 130/161 2023-12-14T18:17:31.522 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 131/161 2023-12-14T18:17:31.732 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 132/161 2023-12-14T18:17:31.945 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 133/161 2023-12-14T18:17:32.535 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:32.846 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:32.886 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:33.371 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 135/161 2023-12-14T18:17:33.774 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 136/161 2023-12-14T18:17:34.041 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 137/161 2023-12-14T18:17:34.254 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 138/161 2023-12-14T18:17:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:34.478 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 139/161 2023-12-14T18:17:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: postgresql-server-10.23-2.module+el8.8.0+19493+5 134/161 2023-12-14T18:17:34.821 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 140/161 2023-12-14T18:17:35.066 INFO:teuthology.orchestra.run.smithi160.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2023-12-14T18:17:35.066 INFO:teuthology.orchestra.run.smithi160.stdout:Installed products updated. 2023-12-14T18:17:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 135/161 2023-12-14T18:17:35.131 INFO:teuthology.orchestra.run.smithi097.stdout: Upgrading : openssl-1:1.1.1k-9.el8_7.x86_64 141/161 2023-12-14T18:17:35.312 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 142/161 2023-12-14T18:17:35.323 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 142/161 2023-12-14T18:17:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : iproute-6.2.0-5.el8_9.x86_64 136/161 2023-12-14T18:17:35.548 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 143/161 2023-12-14T18:17:35.756 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 137/161 2023-12-14T18:17:35.786 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : flex-2.6.1-9.el8.x86_64 144/161 2023-12-14T18:17:35.879 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 144/161 2023-12-14T18:17:35.952 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 138/161 2023-12-14T18:17:36.112 INFO:teuthology.orchestra.run.smithi097.stdout: Installing : bison-3.0.4-10.el8.x86_64 145/161 2023-12-14T18:17:36.126 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 145/161 2023-12-14T18:17:36.168 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 139/161 2023-12-14T18:17:36.346 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout:Upgraded: 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-libs-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-14T18:17:36.347 INFO:teuthology.orchestra.run.smithi160.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout:Installed: 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: automake-1.16.1-8.el8.noarch 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: bison-3.0.4-10.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.348 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-14T18:17:36.349 INFO:teuthology.orchestra.run.smithi160.stdout: flex-2.6.1-9.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: jq-1.6-7.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: libpq-13.11-1.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-14T18:17:36.350 INFO:teuthology.orchestra.run.smithi160.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-14T18:17:36.351 INFO:teuthology.orchestra.run.smithi160.stdout: openssl-devel-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-contrib-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: postgresql-server-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: python3-ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.352 INFO:teuthology.orchestra.run.smithi160.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-14T18:17:36.353 INFO:teuthology.orchestra.run.smithi160.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-portend-2.6-1.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-14T18:17:36.354 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-14T18:17:36.355 INFO:teuthology.orchestra.run.smithi160.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi160.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-14T18:17:36.356 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 146/161 2023-12-14T18:17:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-14T18:17:36.357 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:17:36.357 INFO:teuthology.orchestra.run.smithi160.stdout:Complete! 2023-12-14T18:17:36.518 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 140/161 2023-12-14T18:17:36.522 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 147/161 2023-12-14T18:17:36.682 DEBUG:teuthology.parallel:result is None 2023-12-14T18:17:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 148/161 2023-12-14T18:17:36.750 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:17:36.874 INFO:teuthology.orchestra.run.smithi093.stdout: Upgrading : openssl-1:1.1.1k-9.el8_7.x86_64 141/161 2023-12-14T18:17:37.049 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:17:37.049 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:17:37.126 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : flex-2.6.1-9.el8.x86_64 142/161 2023-12-14T18:17:37.215 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 142/161 2023-12-14T18:17:37.291 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:17:37.291 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:17:37.443 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : bison-3.0.4-10.el8.x86_64 143/161 2023-12-14T18:17:37.457 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 143/161 2023-12-14T18:17:37.539 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:17:37.659 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : NetworkManager-initscripts-updown-1:1.40.16-9.el 144/161 2023-12-14T18:17:37.669 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 144/161 2023-12-14T18:17:37.747 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 152/161 2023-12-14T18:17:37.829 INFO:teuthology.orchestra.run.smithi093.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 145/161 2023-12-14T18:17:37.955 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 153/161 2023-12-14T18:17:37.993 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : elfutils-libs-0.186-1.el8.x86_64 146/161 2023-12-14T18:17:38.203 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : openssl-1:1.1.1k-6.el8_5.x86_64 147/161 2023-12-14T18:17:38.239 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 154/161 2023-12-14T18:17:38.239 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:17:38.406 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 148/161 2023-12-14T18:17:38.406 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:17:38.523 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:17:38.523 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:17:38.654 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 149/161 2023-12-14T18:17:38.654 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:17:38.806 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:17:38.806 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:17:38.897 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 150/161 2023-12-14T18:17:38.897 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:17:39.097 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:17:39.098 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:17:39.157 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 151/161 2023-12-14T18:17:39.381 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:17:39.381 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:17:39.420 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : iproute-5.15.0-4.el8.x86_64 152/161 2023-12-14T18:17:39.605 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:17:39.640 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : util-linux-user-2.32.1-35.el8.x86_64 153/161 2023-12-14T18:17:39.789 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 160/161 2023-12-14T18:17:39.789 INFO:teuthology.orchestra.run.smithi097.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:17:39.895 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : util-linux-2.32.1-35.el8.x86_64 154/161 2023-12-14T18:17:39.895 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:17:40.145 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libfdisk-2.32.1-35.el8.x86_64 155/161 2023-12-14T18:17:40.145 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:17:40.403 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libmount-2.32.1-35.el8.x86_64 156/161 2023-12-14T18:17:40.403 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:17:40.703 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:17:40.704 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 1/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 3/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 4/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 5/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 6/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 7/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 8/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:17:40.763 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 13/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 14/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 15/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 16/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 17/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 18/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 20/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 21/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 22/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 23/161 2023-12-14T18:17:40.764 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 24/161 2023-12-14T18:17:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 25/161 2023-12-14T18:17:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 26/161 2023-12-14T18:17:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 27/161 2023-12-14T18:17:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 28/161 2023-12-14T18:17:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 29/161 2023-12-14T18:17:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 30/161 2023-12-14T18:17:40.765 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 31/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 32/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 33/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 34/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 35/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 36/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 37/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 38/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/161 2023-12-14T18:17:40.777 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/161 2023-12-14T18:17:40.778 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/161 2023-12-14T18:17:40.779 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 72/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 73/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 74/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 75/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 76/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 77/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : bison-3.0.4-10.el8.x86_64 78/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : flex-2.6.1-9.el8.x86_64 79/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 80/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 81/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 82/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 83/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 84/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 85/161 2023-12-14T18:17:40.780 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 86/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 87/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 88/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 89/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 90/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 91/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 92/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 93/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 94/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 95/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 96/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 97/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : postgresql-server-10.23-2.module+el8.8.0+19493+5 98/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : automake-1.16.1-8.el8.noarch 99/161 2023-12-14T18:17:40.781 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libpq-13.11-1.el8.x86_64 100/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : jq-1.6-7.el8.x86_64 101/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 102/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 103/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 104/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 105/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 106/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 107/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 108/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 109/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 110/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 111/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 112/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 113/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 114/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 115/161 2023-12-14T18:17:40.782 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 116/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 117/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8_7.x86_64 118/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 119/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 120/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 121/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 122/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 123/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 124/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 125/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 126/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 127/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 128/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 129/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 130/161 2023-12-14T18:17:40.783 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 132/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:1.1.1k-9.el8_7.x86_64 134/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 135/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8_7.x86_64 136/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 138/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 139/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 140/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 141/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 143/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 145/161 2023-12-14T18:17:40.784 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 147/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 149/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 151/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 154/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 155/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 156/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 158/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:17:40.785 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2023-12-14T18:17:40.953 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libuuid-2.32.1-35.el8.x86_64 158/161 2023-12-14T18:17:40.953 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:17:41.220 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: libsmartcols-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:17:41.378 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : elfutils-libelf-0.186-1.el8.x86_64 160/161 2023-12-14T18:17:41.378 INFO:teuthology.orchestra.run.smithi093.stdout: Cleanup : openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:17:42.365 INFO:teuthology.orchestra.run.smithi093.stdout: Running scriptlet: openssl-libs-1:1.1.1k-6.el8_5.x86_64 161/161 2023-12-14T18:17:42.365 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 1/161 2023-12-14T18:17:42.365 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2/161 2023-12-14T18:17:42.365 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 3/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 4/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-immutable-object-cache-2:18.0.0-3905.g1faa3 5/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 6/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 7/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 8/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 9/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 10/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 11/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 12/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 13/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_ 14/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 15/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 16/161 2023-12-14T18:17:42.366 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_6 17/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86 18/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 19/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el 20/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-ceph-common-2:18.0.0-3905.g1faa3533.el8. 21/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_6 22/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 23/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 24/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 25/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 26/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 27/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 28/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533. 29/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noa 30/161 2023-12-14T18:17:42.367 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.n 31/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-diskprediction-local-2:18.0.0-3905.g1fa 32/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el 33/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 34/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.e 35/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 36/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 37/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 38/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 39/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 40/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 41/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 42/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 43/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 44/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 45/161 2023-12-14T18:17:42.368 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 46/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 47/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 48/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 49/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 50/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 51/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 52/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 53/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 54/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 55/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 56/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-portend-2.6-1.el8.noarch 57/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 58/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 59/161 2023-12-14T18:17:42.369 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 60/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 61/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 62/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 63/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 64/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 65/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 66/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 67/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 68/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 69/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 70/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 71/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libverto-devel-0.3.0-5.el8.x86_64 72/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 73/161 2023-12-14T18:17:42.370 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 74/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 75/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 77/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 78/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-devel-10.32-2.el8.x86_64 79/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf16-10.32-2.el8.x86_64 80/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : pcre2-utf32-10.32-2.el8.x86_64 81/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libselinux-devel-2.9-5.el8.x86_64 82/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 83/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 84/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : zlib-devel-1.2.11-18.el8_5.x86_64 85/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libcom_err-devel-1.45.6-4.el8.x86_64 86/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8_7.x86_64 87/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 88/161 2023-12-14T18:17:42.371 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 89/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 90/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 91/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 92/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 93/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 94/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 95/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 96/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 97/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : krb5-devel-1.18.2-26.el8_9.x86_64 98/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 99/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 100/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 101/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 102/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 103/161 2023-12-14T18:17:42.372 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 104/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : bison-3.0.4-10.el8.x86_64 105/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : flex-2.6.1-9.el8.x86_64 106/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 107/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 108/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 109/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 110/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : uuid-1.6.2-43.el8.x86_64 111/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 112/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd 113/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 114/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 115/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 116/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 117/161 2023-12-14T18:17:42.373 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 118/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 119/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 120/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 121/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 122/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-10.23-2.module+el8.8.0+19493+5804baf6 123/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-contrib-10.23-2.module+el8.8.0+19493+ 124/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : postgresql-server-10.23-2.module+el8.8.0+19493+5 125/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : automake-1.16.1-8.el8.noarch 126/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libpq-13.11-1.el8.x86_64 127/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : jq-1.6-7.el8.x86_64 128/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 129/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 130/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 131/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 132/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 133/161 2023-12-14T18:17:42.374 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-1:1.1.1k-9.el8_7.x86_64 134/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-1:1.1.1k-6.el8_5.x86_64 135/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-libs-1:1.1.1k-9.el8_7.x86_64 136/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : openssl-libs-1:1.1.1k-6.el8_5.x86_64 137/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 138/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libelf-0.186-1.el8.x86_64 139/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 140/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : elfutils-libs-0.186-1.el8.x86_64 141/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libmount-2.32.1-35.el8.x86_64 143/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libsmartcols-2.32.1-35.el8.x86_64 145/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libuuid-2.32.1-35.el8.x86_64 147/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : util-linux-2.32.1-35.el8.x86_64 149/161 2023-12-14T18:17:42.375 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : util-linux-user-2.32.1-35.el8.x86_64 151/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-6.2.0-5.el8_9.x86_64 154/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : iproute-5.15.0-4.el8.x86_64 155/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 156/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libblkid-2.32.1-35.el8.x86_64 157/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 158/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : libfdisk-2.32.1-35.el8.x86_64 159/161 2023-12-14T18:17:42.376 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 160/161 2023-12-14T18:17:50.791 INFO:teuthology.orchestra.run.smithi097.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2023-12-14T18:17:50.791 INFO:teuthology.orchestra.run.smithi097.stdout:Installed products updated. 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout:Upgraded: 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.384 INFO:teuthology.orchestra.run.smithi097.stdout: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-libs-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout:Installed: 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: automake-1.16.1-8.el8.noarch 2023-12-14T18:17:51.385 INFO:teuthology.orchestra.run.smithi097.stdout: bison-3.0.4-10.el8.x86_64 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.386 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: flex-2.6.1-9.el8.x86_64 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-14T18:17:51.387 INFO:teuthology.orchestra.run.smithi097.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: jq-1.6-7.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: libpq-13.11-1.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.388 INFO:teuthology.orchestra.run.smithi097.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: openssl-devel-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:17:51.389 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-contrib-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: postgresql-server-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-14T18:17:51.390 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-14T18:17:51.391 INFO:teuthology.orchestra.run.smithi097.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-portend-2.6-1.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-14T18:17:51.392 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-14T18:17:51.393 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout: rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout: 2023-12-14T18:17:51.394 INFO:teuthology.orchestra.run.smithi097.stdout:Complete! 2023-12-14T18:17:51.581 DEBUG:teuthology.parallel:result is None 2023-12-14T18:17:56.589 INFO:teuthology.orchestra.run.smithi093.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 161/161 2023-12-14T18:17:56.589 INFO:teuthology.orchestra.run.smithi093.stdout:Installed products updated. 2023-12-14T18:17:57.184 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:17:57.184 INFO:teuthology.orchestra.run.smithi093.stdout:Upgraded: 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: iproute-6.2.0-5.el8_9.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: libblkid-2.32.1-43.el8.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: libmount-2.32.1-43.el8.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: librados2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: librbd1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-2.32.1-43.el8.x86_64 2023-12-14T18:17:57.185 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-libs-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: util-linux-2.32.1-43.el8.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout:Installed: 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: automake-1.16.1-8.el8.noarch 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: bison-3.0.4-10.el8.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-base-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.186 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-grafana-dashboards-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-immutable-object-cache-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mds-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-dashboard-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-diskprediction-local-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-modules-core-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mgr-rook-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-mon-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-osd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-prometheus-alerts-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-radosgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.187 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-selinux-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-test-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: ceph-volume-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: cephadm-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: cephfs-top-2:18.0.0-3905.g1faa3533.el8.noarch 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: flex-2.6.1-9.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: fuse-2.9.7-17.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: jq-1.6-7.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: krb5-devel-1.18.2-26.el8_9.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.188 INFO:teuthology.orchestra.run.smithi093.stdout: libcephfs2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: libcephsqlite-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: libcom_err-devel-1.45.6-4.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: liboath-2.6.2-3.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: libpq-13.11-1.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: librados-devel-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: libradosstriper1-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: librgw2-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: libselinux-devel-2.9-5.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-12-14T18:17:57.189 INFO:teuthology.orchestra.run.smithi093.stdout: libtool-2.4.6-25.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: libunwind-1.3.1-3.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: libverto-devel-0.3.0-5.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-0.3.15-6.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: openblas-threads-0.3.15-6.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: openssl-devel-1:1.1.1k-9.el8_7.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-devel-10.32-2.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf16-10.32-2.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: pcre2-utf32-10.32-2.el8.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-contrib-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: postgresql-server-10.23-2.module+el8.8.0+19493+5804baf6.x86_64 2023-12-14T18:17:57.190 INFO:teuthology.orchestra.run.smithi093.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-babel-2.5.1-7.el8.noarch 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-argparse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-ceph-common-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cephfs-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-6.2-6.el8.noarch 2023-12-14T18:17:57.191 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-mako-1.0.6-14.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-12-14T18:17:57.192 INFO:teuthology.orchestra.run.smithi093.stdout: python3-portend-2.6-1.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pycparser-2.14-14.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-pytz-2017.2-11.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rados-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rgw-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-routes-2.4.1-12.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-rsa-4.9-2.el8.noarch 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-scipy-1.0.0-21.module+el8.5.0+10916+41bd434d.x86_64 2023-12-14T18:17:57.193 INFO:teuthology.orchestra.run.smithi093.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-fuse-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-mirror-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: rbd-nbd-2:18.0.0-3905.g1faa3533.el8.x86_64 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: socat-1.7.4.1-1.el8.x86_64 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: thrift-0.13.0-2.el8.x86_64 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: uuid-1.6.2-43.el8.x86_64 2023-12-14T18:17:57.194 INFO:teuthology.orchestra.run.smithi093.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-12-14T18:17:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-12-14T18:17:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: zlib-devel-1.2.11-18.el8_5.x86_64 2023-12-14T18:17:57.195 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:17:57.195 INFO:teuthology.orchestra.run.smithi093.stdout:Complete! 2023-12-14T18:17:57.401 DEBUG:teuthology.parallel:result is None 2023-12-14T18:17:57.402 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:17:57.544 DEBUG:teuthology.orchestra.run.smithi093:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-14T18:17:57.567 INFO:teuthology.orchestra.run.smithi093.stdout:18.0.0-3905.g1faa3533.el8 2023-12-14T18:17:57.568 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g1faa3533.el8 2023-12-14T18:17:57.568 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g1faa3533 is installed. 2023-12-14T18:17:57.569 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:17:57.781 DEBUG:teuthology.orchestra.run.smithi097:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-14T18:17:57.809 INFO:teuthology.orchestra.run.smithi097.stdout:18.0.0-3905.g1faa3533.el8 2023-12-14T18:17:57.809 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g1faa3533.el8 2023-12-14T18:17:57.810 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g1faa3533 is installed. 2023-12-14T18:17:57.811 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:17:57.949 DEBUG:teuthology.orchestra.run.smithi160:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-12-14T18:17:57.978 INFO:teuthology.orchestra.run.smithi160.stdout:18.0.0-3905.g1faa3533.el8 2023-12-14T18:17:57.978 INFO:teuthology.packaging:The installed version of ceph is 18.0.0-3905.g1faa3533.el8 2023-12-14T18:17:57.978 INFO:teuthology.task.install:The correct ceph version 18.0.0-3905.g1faa3533 is installed. 2023-12-14T18:17:57.980 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-14T18:17:57.980 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:17:57.980 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-14T18:17:58.022 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:17:58.022 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-14T18:17:58.069 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:17:58.070 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-14T18:17:58.118 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-14T18:17:58.118 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:17:58.118 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/daemon-helper 2023-12-14T18:17:58.145 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-14T18:17:58.212 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:17:58.212 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/daemon-helper 2023-12-14T18:17:58.243 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-14T18:17:58.313 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:17:58.313 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/daemon-helper 2023-12-14T18:17:58.346 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-14T18:17:58.416 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-14T18:17:58.417 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:17:58.417 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-14T18:17:58.443 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-14T18:17:58.510 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:17:58.510 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-14T18:17:58.541 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-14T18:17:58.611 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:17:58.611 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-14T18:17:58.641 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-14T18:17:58.713 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-14T18:17:58.713 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:17:58.713 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/usr/bin/stdin-killer 2023-12-14T18:17:58.740 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-14T18:17:58.806 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:17:58.806 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/usr/bin/stdin-killer 2023-12-14T18:17:58.836 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-14T18:17:58.905 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:17:58.905 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/usr/bin/stdin-killer 2023-12-14T18:17:58.935 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-14T18:17:59.006 INFO:teuthology.run_tasks:Running task cephadm... 2023-12-14T18:17:59.078 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-12-14T18:17:59.078 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:17:59.079 INFO:tasks.cephadm:Cluster fsid is 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:17:59.079 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2023-12-14T18:17:59.079 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.93', 'mon.b': '172.21.15.97', 'mon.c': '172.21.15.160'} 2023-12-14T18:17:59.079 INFO:tasks.cephadm:First mon is mon.a on smithi093 2023-12-14T18:17:59.079 INFO:tasks.cephadm:First mgr is x 2023-12-14T18:17:59.079 INFO:tasks.cephadm:Normalizing hostnames... 2023-12-14T18:17:59.080 DEBUG:teuthology.orchestra.run.smithi093:> sudo hostname $(hostname -s) 2023-12-14T18:17:59.107 DEBUG:teuthology.orchestra.run.smithi097:> sudo hostname $(hostname -s) 2023-12-14T18:17:59.137 DEBUG:teuthology.orchestra.run.smithi160:> sudo hostname $(hostname -s) 2023-12-14T18:17:59.170 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2023-12-14T18:17:59.170 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:17:59.389 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1faa353368ad4f5aeb56d7107b638336f58659ed', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic/74952/', 'root_build_cause': 'SCMTRIGGER', 'version': '18.0.0-3905-g1faa3533', 'node_name': '172.21.5.31+adami01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos8,DIST=centos8,MACHINE_SIZE=gigantic', 'package_manager_version': '18.0.0-3905.g1faa3533'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/', 'distro_codename': None, 'modified': '2023-12-13 17:15:35.607942', 'distro_version': '8', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-choffman-62793', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/flavors/default/', 'archs': ['x86_64', 'source'], 'distro': 'centos'}] 2023-12-14T18:17:59.533 INFO:tasks.util.chacra:got chacra host 3.chacra.ceph.com, ref wip-choffman-62793, sha1 1faa353368ad4f5aeb56d7107b638336f58659ed from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F8%2Fx86_64&flavor=default&sha1=1faa353368ad4f5aeb56d7107b638336f58659ed 2023-12-14T18:17:59.534 INFO:tasks.cephadm:Discovered cachra url: https://3.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/x86_64/flavors/default/cephadm 2023-12-14T18:17:59.534 DEBUG:teuthology.orchestra.run.smithi093:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:00.066 INFO:teuthology.orchestra.run.smithi093.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Dec 14 18:18 /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:00.067 DEBUG:teuthology.orchestra.run.smithi097:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:00.553 INFO:teuthology.orchestra.run.smithi097.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Dec 14 18:18 /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:00.554 DEBUG:teuthology.orchestra.run.smithi160:> curl --silent -L https://3.chacra.ceph.com/binaries/ceph/wip-choffman-62793/1faa353368ad4f5aeb56d7107b638336f58659ed/centos/8/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:00.932 INFO:teuthology.orchestra.run.smithi160.stdout:-rw-rw-r--. 1 ubuntu ubuntu 384133 Dec 14 18:18 /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:00.932 DEBUG:teuthology.orchestra.run.smithi093:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:00.955 DEBUG:teuthology.orchestra.run.smithi097:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:00.977 DEBUG:teuthology.orchestra.run.smithi160:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2023-12-14T18:18:01.020 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed on all hosts... 2023-12-14T18:18:01.021 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed pull 2023-12-14T18:18:01.023 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed pull 2023-12-14T18:18:01.026 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed pull 2023-12-14T18:18:02.389 INFO:teuthology.orchestra.run.smithi160.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:18:03.116 INFO:teuthology.orchestra.run.smithi097.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:18:03.594 INFO:teuthology.orchestra.run.smithi093.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:18:34.525 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:18:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_version": "ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev)", 2023-12-14T18:18:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "image_id": "0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0", 2023-12-14T18:18:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "repo_digests": [ 2023-12-14T18:18:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4f00bb7c6b97266b68e12e45f4e426e52b86010cb97d3e773903241deee4098b" 2023-12-14T18:18:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:18:34.526 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:18:39.078 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2023-12-14T18:18:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_version": "ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev)", 2023-12-14T18:18:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "image_id": "0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0", 2023-12-14T18:18:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "repo_digests": [ 2023-12-14T18:18:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4f00bb7c6b97266b68e12e45f4e426e52b86010cb97d3e773903241deee4098b" 2023-12-14T18:18:39.078 INFO:teuthology.orchestra.run.smithi160.stdout: ] 2023-12-14T18:18:39.078 INFO:teuthology.orchestra.run.smithi160.stdout:} 2023-12-14T18:18:39.315 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-12-14T18:18:39.315 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_version": "ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev)", 2023-12-14T18:18:39.315 INFO:teuthology.orchestra.run.smithi097.stdout: "image_id": "0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0", 2023-12-14T18:18:39.316 INFO:teuthology.orchestra.run.smithi097.stdout: "repo_digests": [ 2023-12-14T18:18:39.316 INFO:teuthology.orchestra.run.smithi097.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:4f00bb7c6b97266b68e12e45f4e426e52b86010cb97d3e773903241deee4098b" 2023-12-14T18:18:39.316 INFO:teuthology.orchestra.run.smithi097.stdout: ] 2023-12-14T18:18:39.316 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-12-14T18:18:39.335 DEBUG:teuthology.orchestra.run.smithi093:> sudo mkdir -p /etc/ceph 2023-12-14T18:18:39.371 DEBUG:teuthology.orchestra.run.smithi097:> sudo mkdir -p /etc/ceph 2023-12-14T18:18:39.400 DEBUG:teuthology.orchestra.run.smithi160:> sudo mkdir -p /etc/ceph 2023-12-14T18:18:39.431 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 777 /etc/ceph 2023-12-14T18:18:39.462 DEBUG:teuthology.orchestra.run.smithi097:> sudo chmod 777 /etc/ceph 2023-12-14T18:18:39.488 DEBUG:teuthology.orchestra.run.smithi160:> sudo chmod 777 /etc/ceph 2023-12-14T18:18:39.516 INFO:tasks.cephadm:Writing seed config... 2023-12-14T18:18:39.517 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2023-12-14T18:18:39.517 INFO:tasks.cephadm: override: [client] debug client = 20 2023-12-14T18:18:39.517 INFO:tasks.cephadm: override: [client] debug ms = 1 2023-12-14T18:18:39.517 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2023-12-14T18:18:39.517 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2023-12-14T18:18:39.517 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2023-12-14T18:18:39.517 INFO:tasks.cephadm: override: [mds] debug mds = 20 2023-12-14T18:18:39.517 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2023-12-14T18:18:39.517 INFO:tasks.cephadm: override: [mds] debug ms = 1 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2023-12-14T18:18:39.518 INFO:tasks.cephadm: override: [mon] debug mon = 20 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [mon] debug ms = 1 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] debug ms = 1 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] debug osd = 20 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2023-12-14T18:18:39.519 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2023-12-14T18:18:39.520 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2023-12-14T18:18:39.520 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:18:39.520 DEBUG:teuthology.orchestra.run.smithi093:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2023-12-14T18:18:39.539 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 1c5ac00e-9aad-11ee-95a5-87774f69a715 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2023-12-14T18:18:39.539 DEBUG:teuthology.orchestra.run.smithi093:mon.a> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.a.service 2023-12-14T18:18:39.582 DEBUG:teuthology.orchestra.run.smithi093:mgr.x> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.x.service 2023-12-14T18:18:39.612 INFO:journalctl@ceph.mon.a.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:18:39.613 INFO:tasks.cephadm:Bootstrapping... 2023-12-14T18:18:39.613 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed -v bootstrap --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.93 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:18:39.642 INFO:journalctl@ceph.mgr.x.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:18:39.865 INFO:teuthology.orchestra.run.smithi093.stdout:-------------------------------------------------------------------------------- 2023-12-14T18:18:39.865 INFO:teuthology.orchestra.run.smithi093.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed', '-v', 'bootstrap', '--fsid', '1c5ac00e-9aad-11ee-95a5-87774f69a715', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.93', '--skip-admin-label'] 2023-12-14T18:18:39.895 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 4.6.1 2023-12-14T18:18:39.895 INFO:teuthology.orchestra.run.smithi093.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2023-12-14T18:18:39.895 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying podman|docker is present... 2023-12-14T18:18:39.927 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 4.6.1 2023-12-14T18:18:39.927 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying lvm2 is present... 2023-12-14T18:18:39.927 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying time synchronization is in place... 2023-12-14T18:18:39.934 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-14T18:18:39.934 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-14T18:18:39.940 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-14T18:18:39.940 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-12-14T18:18:39.946 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout enabled 2023-12-14T18:18:39.952 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout active 2023-12-14T18:18:39.952 INFO:teuthology.orchestra.run.smithi093.stdout:Unit chronyd.service is enabled and running 2023-12-14T18:18:39.952 INFO:teuthology.orchestra.run.smithi093.stdout:Repeating the final host check... 2023-12-14T18:18:39.982 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 4.6.1 2023-12-14T18:18:39.982 INFO:teuthology.orchestra.run.smithi093.stdout:podman (/bin/podman) version 4.6.1 is present 2023-12-14T18:18:39.982 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl is present 2023-12-14T18:18:39.982 INFO:teuthology.orchestra.run.smithi093.stdout:lvcreate is present 2023-12-14T18:18:39.988 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2023-12-14T18:18:39.988 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2023-12-14T18:18:39.994 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2023-12-14T18:18:39.994 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-12-14T18:18:40.000 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout enabled 2023-12-14T18:18:40.006 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout active 2023-12-14T18:18:40.006 INFO:teuthology.orchestra.run.smithi093.stdout:Unit chronyd.service is enabled and running 2023-12-14T18:18:40.006 INFO:teuthology.orchestra.run.smithi093.stdout:Host looks OK 2023-12-14T18:18:40.006 INFO:teuthology.orchestra.run.smithi093.stdout:Cluster fsid: 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:18:40.006 INFO:teuthology.orchestra.run.smithi093.stdout:Acquiring lock 140048011880936 on /run/cephadm/1c5ac00e-9aad-11ee-95a5-87774f69a715.lock 2023-12-14T18:18:40.006 INFO:teuthology.orchestra.run.smithi093.stdout:Lock 140048011880936 acquired on /run/cephadm/1c5ac00e-9aad-11ee-95a5-87774f69a715.lock 2023-12-14T18:18:40.007 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying IP 172.21.15.93 port 3300 ... 2023-12-14T18:18:40.007 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying IP 172.21.15.93 port 6789 ... 2023-12-14T18:18:40.007 INFO:teuthology.orchestra.run.smithi093.stdout:Base mon IP(s) is [172.21.15.93:3300, 172.21.15.93:6789], mon addrv is [v2:172.21.15.93:3300,v1:172.21.15.93:6789] 2023-12-14T18:18:40.011 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp metric 100 2023-12-14T18:18:40.011 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.93 metric 100 2023-12-14T18:18:40.014 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2023-12-14T18:18:40.014 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2023-12-14T18:18:40.014 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2023-12-14T18:18:40.017 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2023-12-14T18:18:40.017 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2023-12-14T18:18:40.017 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-14T18:18:40.017 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2023-12-14T18:18:40.017 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout inet6 fe80::1355:f7c1:e067:88d9/64 scope link noprefixroute 2023-12-14T18:18:40.017 INFO:teuthology.orchestra.run.smithi093.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2023-12-14T18:18:40.018 INFO:teuthology.orchestra.run.smithi093.stdout:Mon IP `172.21.15.93` is in CIDR network `172.21.0.0/20` 2023-12-14T18:18:40.018 INFO:teuthology.orchestra.run.smithi093.stdout:Mon IP `172.21.15.93` is in CIDR network `172.21.0.0/20` 2023-12-14T18:18:40.018 INFO:teuthology.orchestra.run.smithi093.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2023-12-14T18:18:40.018 INFO:teuthology.orchestra.run.smithi093.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2023-12-14T18:18:40.019 INFO:teuthology.orchestra.run.smithi093.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:18:40.450 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stdout 0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0 2023-12-14T18:18:40.450 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed... 2023-12-14T18:18:40.450 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Getting image source signatures 2023-12-14T18:18:40.450 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Copying blob sha256:a83bc5fcf8de435b160e58e3bb5830aa8038f139ef00a293340a26641aa5a474 2023-12-14T18:18:40.450 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Copying blob sha256:46af8f5390d4e94fc57efb422ccb97bb53dfe5b948546bfc191b46557eb2dbd9 2023-12-14T18:18:40.451 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Copying config sha256:0caf899250bd8e7e64baa66bc41be9a670e1a24f023c9b5250d24ffadc35c5c0 2023-12-14T18:18:40.451 INFO:teuthology.orchestra.run.smithi093.stdout:/bin/podman: stderr Writing manifest to image destination 2023-12-14T18:18:41.059 INFO:teuthology.orchestra.run.smithi093.stdout:ceph: stdout ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev) 2023-12-14T18:18:41.059 INFO:teuthology.orchestra.run.smithi093.stdout:Ceph version: ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev) 2023-12-14T18:18:41.059 INFO:teuthology.orchestra.run.smithi093.stdout:Extracting ceph user uid/gid from container image... 2023-12-14T18:18:41.697 INFO:teuthology.orchestra.run.smithi093.stdout:stat: stdout 167 167 2023-12-14T18:18:41.697 INFO:teuthology.orchestra.run.smithi093.stdout:Creating initial keys... 2023-12-14T18:18:42.280 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph-authtool: stdout AQABR3tlBc1cNxAAFkVwAoOuRHq5ty6GM803+w== 2023-12-14T18:18:42.903 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph-authtool: stdout AQACR3tlmJ8KHxAAuV3AyFnIQ8TeV4sWcIzuDQ== 2023-12-14T18:18:43.490 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph-authtool: stdout AQADR3tlKg0sCBAA+jHSrtpoJQkLkJ1l6+ReDw== 2023-12-14T18:18:43.491 INFO:teuthology.orchestra.run.smithi093.stdout:Creating initial monmap... 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = pacific 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout:monmaptool for a [v2:172.21.15.93:3300,v1:172.21.15.93:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout:setting min_mon_release = pacific 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: set fsid to 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:18:44.155 INFO:teuthology.orchestra.run.smithi093.stdout:Creating mon... 2023-12-14T18:18:44.794 INFO:teuthology.orchestra.run.smithi093.stdout:create mon.a on 2023-12-14T18:18:44.893 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Removed /etc/systemd/system/multi-user.target.wants/ceph.target. 2023-12-14T18:18:44.984 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2023-12-14T18:18:45.076 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715.target → /etc/systemd/system/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715.target. 2023-12-14T18:18:45.076 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715.target → /etc/systemd/system/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715.target. 2023-12-14T18:18:45.198 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.a 2023-12-14T18:18:45.198 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.a.service: Unit ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.a.service not loaded. 2023-12-14T18:18:45.286 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715.target.wants/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.a.service → /etc/systemd/system/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@.service. 2023-12-14T18:18:47.235 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-14T18:18:47.235 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-12-14T18:18:47.242 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-14T18:18:47.242 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-12-14T18:18:47.242 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-12-14T18:18:47.242 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to enable service . firewalld.service is not available 2023-12-14T18:18:47.243 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mon to start... 2023-12-14T18:18:47.243 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mon... 2023-12-14T18:18:47.403 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:47 smithi093 bash[41546]: 6df8045b17a305e20689bca6f6b685f4f57bebc3c77472417a0d686322c029c6 2023-12-14T18:18:47.404 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:47 smithi093 systemd[1]: Started Ceph mon.a for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:18:48.084 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout cluster: 2023-12-14T18:18:48.084 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout id: 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:18:48.084 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2023-12-14T18:18:48.084 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:18:48.084 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout services: 2023-12-14T18:18:48.084 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.644399s) 2023-12-14T18:18:48.084 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2023-12-14T18:18:48.084 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout data: 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout pgs: 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.651+0000 7f5c7329e700 1 Processor -- start 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.651+0000 7f5c7329e700 1 -- start start 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.651+0000 7f5c7329e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c07a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.651+0000 7f5c7329e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c07a8a0 con 0x7f5c6c07be70 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.651+0000 7f5c7103a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c07a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.651+0000 7f5c7103a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c07a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35492/0 (socket says 172.21.15.93:35492) 2023-12-14T18:18:48.085 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.651+0000 7f5c7103a700 1 -- 172.21.15.93:0/3102536765 learned_addr learned my addr 172.21.15.93:0/3102536765 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c7103a700 1 -- 172.21.15.93:0/3102536765 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c6c07a9e0 con 0x7f5c6c07be70 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c7103a700 1 --2- 172.21.15.93:0/3102536765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c07a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c680098a0 tx=0x7f5c68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=332797b40c1537c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c5ffff700 1 -- 172.21.15.93:0/3102536765 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c68004160 con 0x7f5c6c07be70 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c5ffff700 1 -- 172.21.15.93:0/3102536765 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c680042c0 con 0x7f5c6c07be70 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c5ffff700 1 -- 172.21.15.93:0/3102536765 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c680045a0 con 0x7f5c6c07be70 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c7329e700 1 -- 172.21.15.93:0/3102536765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 msgr2=0x7f5c6c07a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c7329e700 1 --2- 172.21.15.93:0/3102536765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c07a2d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f5c680098a0 tx=0x7f5c68009bb0 comp rx=0 tx=0).stop 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c7329e700 1 -- 172.21.15.93:0/3102536765 shutdown_connections 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c7329e700 1 --2- 172.21.15.93:0/3102536765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c07a2d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.652+0000 7f5c7329e700 1 -- 172.21.15.93:0/3102536765 >> 172.21.15.93:0/3102536765 conn(0x7f5c6c100fd0 msgr2=0x7f5c6c103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.653+0000 7f5c7329e700 1 -- 172.21.15.93:0/3102536765 shutdown_connections 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.653+0000 7f5c7329e700 1 -- 172.21.15.93:0/3102536765 wait complete. 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.653+0000 7f5c7329e700 1 Processor -- start 2023-12-14T18:18:48.086 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.653+0000 7f5c7329e700 1 -- start start 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c7329e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c19b180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c7329e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c19b6c0 con 0x7f5c6c07be70 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c7103a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c19b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c7103a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c19b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35494/0 (socket says 172.21.15.93:35494) 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c7103a700 1 -- 172.21.15.93:0/1300690600 learned_addr learned my addr 172.21.15.93:0/1300690600 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c7103a700 1 -- 172.21.15.93:0/1300690600 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c68009550 con 0x7f5c6c07be70 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c7103a700 1 --2- 172.21.15.93:0/1300690600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c19b180 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009fb0 tx=0x7f5c68004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c5e7fc700 1 -- 172.21.15.93:0/1300690600 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c68003dd0 con 0x7f5c6c07be70 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c5e7fc700 1 -- 172.21.15.93:0/1300690600 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c68011630 con 0x7f5c6c07be70 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c7329e700 1 -- 172.21.15.93:0/1300690600 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c07aee0 con 0x7f5c6c07be70 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c5e7fc700 1 -- 172.21.15.93:0/1300690600 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c68011910 con 0x7f5c6c07be70 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c7329e700 1 -- 172.21.15.93:0/1300690600 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c07ad50 con 0x7f5c6c07be70 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.654+0000 7f5c5e7fc700 1 -- 172.21.15.93:0/1300690600 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f5c6801c460 con 0x7f5c6c07be70 2023-12-14T18:18:48.087 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.655+0000 7f5c5e7fc700 1 -- 172.21.15.93:0/1300690600 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f5c6801a7e0 con 0x7f5c6c07be70 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.655+0000 7f5c7329e700 1 -- 172.21.15.93:0/1300690600 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c6c194fb0 con 0x7f5c6c07be70 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.657+0000 7f5c5e7fc700 1 -- 172.21.15.93:0/1300690600 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f5c6801aaf0 con 0x7f5c6c07be70 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.727+0000 7f5c7329e700 1 -- 172.21.15.93:0/1300690600 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f5c6c02cfe0 con 0x7f5c6c07be70 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.727+0000 7f5c5e7fc700 1 -- 172.21.15.93:0/1300690600 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+317 (secure 0 0 0) 0x7f5c6801ac90 con 0x7f5c6c07be70 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.728+0000 7f5c7329e700 1 -- 172.21.15.93:0/1300690600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 msgr2=0x7f5c6c19b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.728+0000 7f5c7329e700 1 --2- 172.21.15.93:0/1300690600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c19b180 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f5c68009fb0 tx=0x7f5c68004750 comp rx=0 tx=0).stop 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.728+0000 7f5c7329e700 1 -- 172.21.15.93:0/1300690600 shutdown_connections 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.728+0000 7f5c7329e700 1 --2- 172.21.15.93:0/1300690600 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c07be70 0x7f5c6c19b180 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.728+0000 7f5c7329e700 1 -- 172.21.15.93:0/1300690600 >> 172.21.15.93:0/1300690600 conn(0x7f5c6c100fd0 msgr2=0x7f5c6c1029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.728+0000 7f5c7329e700 1 -- 172.21.15.93:0/1300690600 shutdown_connections 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:47.728+0000 7f5c7329e700 1 -- 172.21.15.93:0/1300690600 wait complete. 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:mon is available 2023-12-14T18:18:48.088 INFO:teuthology.orchestra.run.smithi093.stdout:Assimilating anything we can from ceph.conf... 2023-12-14T18:18:48.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:48 smithi093 ceph-mon[41584]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-14T18:18:48.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:48 smithi093 ceph-mon[41584]: monmap e1: 1 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0]} removed_ranks: {} 2023-12-14T18:18:48.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:48 smithi093 ceph-mon[41584]: fsmap 2023-12-14T18:18:48.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:48 smithi093 ceph-mon[41584]: osdmap e1: 0 total, 0 up, 0 in 2023-12-14T18:18:48.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:48 smithi093 ceph-mon[41584]: mgrmap e1: no daemons active 2023-12-14T18:18:48.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:48 smithi093 ceph-mon[41584]: from='client.? 172.21.15.93:0/1300690600' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2023-12-14T18:18:48.863 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:18:48.864 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [global] 2023-12-14T18:18:48.872 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout fsid = 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:18:48.872 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.93:3300,v1:172.21.15.93:6789] 2023-12-14T18:18:48.872 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-14T18:18:48.872 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-14T18:18:48.872 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-14T18:18:48.872 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-14T18:18:48.872 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:18:48.872 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [osd] 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.447+0000 7f9fdb82d700 1 Processor -- start 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.447+0000 7f9fdb82d700 1 -- start start 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.447+0000 7f9fdb82d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd41088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.447+0000 7f9fdb82d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4079e20 con 0x7f9fd41064d0 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.448+0000 7f9fd95c9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd41088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:48.873 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.448+0000 7f9fd95c9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd41088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35496/0 (socket says 172.21.15.93:35496) 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.448+0000 7f9fd95c9700 1 -- 172.21.15.93:0/207481147 learned_addr learned my addr 172.21.15.93:0/207481147 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.448+0000 7f9fd95c9700 1 -- 172.21.15.93:0/207481147 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd4108df0 con 0x7f9fd41064d0 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.448+0000 7f9fd95c9700 1 --2- 172.21.15.93:0/207481147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd41088b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40098a0 tx=0x7f9fc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6653782bcaa0b047 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.448+0000 7f9fcbfff700 1 -- 172.21.15.93:0/207481147 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fc4004160 con 0x7f9fd41064d0 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.449+0000 7f9fcbfff700 1 -- 172.21.15.93:0/207481147 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc40042c0 con 0x7f9fd41064d0 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.449+0000 7f9fcbfff700 1 -- 172.21.15.93:0/207481147 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fc40045a0 con 0x7f9fd41064d0 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.449+0000 7f9fdb82d700 1 -- 172.21.15.93:0/207481147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 msgr2=0x7f9fd41088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.449+0000 7f9fdb82d700 1 --2- 172.21.15.93:0/207481147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd41088b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f9fc40098a0 tx=0x7f9fc4009bb0 comp rx=0 tx=0).stop 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.449+0000 7f9fdb82d700 1 -- 172.21.15.93:0/207481147 shutdown_connections 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.449+0000 7f9fdb82d700 1 --2- 172.21.15.93:0/207481147 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd41088b0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.449+0000 7f9fdb82d700 1 -- 172.21.15.93:0/207481147 >> 172.21.15.93:0/207481147 conn(0x7f9fd40ffed0 msgr2=0x7f9fd4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.449+0000 7f9fdb82d700 1 -- 172.21.15.93:0/207481147 shutdown_connections 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.449+0000 7f9fdb82d700 1 -- 172.21.15.93:0/207481147 wait complete. 2023-12-14T18:18:48.874 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.450+0000 7f9fdb82d700 1 Processor -- start 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.450+0000 7f9fdb82d700 1 -- start start 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.450+0000 7f9fdb82d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd4198f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.450+0000 7f9fdb82d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd4199470 con 0x7f9fd41064d0 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.450+0000 7f9fd95c9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd4198f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.450+0000 7f9fd95c9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd4198f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35504/0 (socket says 172.21.15.93:35504) 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.450+0000 7f9fd95c9700 1 -- 172.21.15.93:0/3043240755 learned_addr learned my addr 172.21.15.93:0/3043240755 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.451+0000 7f9fd95c9700 1 -- 172.21.15.93:0/3043240755 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc4009550 con 0x7f9fd41064d0 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.451+0000 7f9fd95c9700 1 --2- 172.21.15.93:0/3043240755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd4198f30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4079260 tx=0x7f9fc4004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.451+0000 7f9fca7fc700 1 -- 172.21.15.93:0/3043240755 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fc4011630 con 0x7f9fd41064d0 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.451+0000 7f9fca7fc700 1 -- 172.21.15.93:0/3043240755 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc4011790 con 0x7f9fd41064d0 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.451+0000 7f9fdb82d700 1 -- 172.21.15.93:0/3043240755 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd4109230 con 0x7f9fd41064d0 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.451+0000 7f9fca7fc700 1 -- 172.21.15.93:0/3043240755 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9fc4011a30 con 0x7f9fd41064d0 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.451+0000 7f9fdb82d700 1 -- 172.21.15.93:0/3043240755 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd41090a0 con 0x7f9fd41064d0 2023-12-14T18:18:48.875 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.451+0000 7f9fca7fc700 1 -- 172.21.15.93:0/3043240755 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f9fc4022070 con 0x7f9fd41064d0 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.451+0000 7f9fca7fc700 1 -- 172.21.15.93:0/3043240755 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f9fc401c450 con 0x7f9fd41064d0 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.452+0000 7f9fdb82d700 1 -- 172.21.15.93:0/3043240755 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd4192dc0 con 0x7f9fd41064d0 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.454+0000 7f9fca7fc700 1 -- 172.21.15.93:0/3043240755 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f9fc401b450 con 0x7f9fd41064d0 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.520+0000 7f9fdb82d700 1 -- 172.21.15.93:0/3043240755 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f9fd4062120 con 0x7f9fd41064d0 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.524+0000 7f9fca7fc700 1 -- 172.21.15.93:0/3043240755 <== mon.0 v2:172.21.15.93:3300/0 7 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9fc401b5f0 con 0x7f9fd41064d0 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.524+0000 7f9fca7fc700 1 -- 172.21.15.93:0/3043240755 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+429 (secure 0 0 0) 0x7f9fd4062120 con 0x7f9fd41064d0 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.525+0000 7f9fdb82d700 1 -- 172.21.15.93:0/3043240755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 msgr2=0x7f9fd4198f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.525+0000 7f9fdb82d700 1 --2- 172.21.15.93:0/3043240755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd4198f30 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4079260 tx=0x7f9fc4004750 comp rx=0 tx=0).stop 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.525+0000 7f9fdb82d700 1 -- 172.21.15.93:0/3043240755 shutdown_connections 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.525+0000 7f9fdb82d700 1 --2- 172.21.15.93:0/3043240755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9fd41064d0 0x7f9fd4198f30 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.525+0000 7f9fdb82d700 1 -- 172.21.15.93:0/3043240755 >> 172.21.15.93:0/3043240755 conn(0x7f9fd40ffed0 msgr2=0x7f9fd4101870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.525+0000 7f9fdb82d700 1 -- 172.21.15.93:0/3043240755 shutdown_connections 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:48.525+0000 7f9fdb82d700 1 -- 172.21.15.93:0/3043240755 wait complete. 2023-12-14T18:18:48.876 INFO:teuthology.orchestra.run.smithi093.stdout:Generating new minimal ceph.conf... 2023-12-14T18:18:49.285 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:49 smithi093 ceph-mon[41584]: from='client.? 172.21.15.93:0/3043240755' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-14T18:18:49.285 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:49 smithi093 ceph-mon[41584]: from='client.? 172.21.15.93:0/3043240755' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2023-12-14T18:18:49.616 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.190+0000 7f2568e6b700 1 Processor -- start 2023-12-14T18:18:49.616 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.190+0000 7f2568e6b700 1 -- start start 2023-12-14T18:18:49.616 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.190+0000 7f2568e6b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f2564105230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:49.616 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.190+0000 7f2568e6b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2564105800 con 0x7f2564104e20 2023-12-14T18:18:49.616 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.190+0000 7f256259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f2564105230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:49.616 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.190+0000 7f256259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f2564105230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35512/0 (socket says 172.21.15.93:35512) 2023-12-14T18:18:49.616 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.190+0000 7f256259c700 1 -- 172.21.15.93:0/552162029 learned_addr learned my addr 172.21.15.93:0/552162029 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.190+0000 7f256259c700 1 -- 172.21.15.93:0/552162029 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2564105b40 con 0x7f2564104e20 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f256259c700 1 --2- 172.21.15.93:0/552162029 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f2564105230 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f25540098a0 tx=0x7f2554009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad10dbb2274e562a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f256159a700 1 -- 172.21.15.93:0/552162029 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2554004160 con 0x7f2564104e20 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f256159a700 1 -- 172.21.15.93:0/552162029 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f25540042c0 con 0x7f2564104e20 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f256159a700 1 -- 172.21.15.93:0/552162029 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2554011a00 con 0x7f2564104e20 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f2568e6b700 1 -- 172.21.15.93:0/552162029 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 msgr2=0x7f2564105230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f2568e6b700 1 --2- 172.21.15.93:0/552162029 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f2564105230 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f25540098a0 tx=0x7f2554009bb0 comp rx=0 tx=0).stop 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f2568e6b700 1 -- 172.21.15.93:0/552162029 shutdown_connections 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f2568e6b700 1 --2- 172.21.15.93:0/552162029 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f2564105230 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f2568e6b700 1 -- 172.21.15.93:0/552162029 >> 172.21.15.93:0/552162029 conn(0x7f25640ff650 msgr2=0x7f2564101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f2568e6b700 1 -- 172.21.15.93:0/552162029 shutdown_connections 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.191+0000 7f2568e6b700 1 -- 172.21.15.93:0/552162029 wait complete. 2023-12-14T18:18:49.617 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.192+0000 7f2568e6b700 1 Processor -- start 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.192+0000 7f2568e6b700 1 -- start start 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.192+0000 7f2568e6b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f25641987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.192+0000 7f2568e6b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2564198d30 con 0x7f2564104e20 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f256259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f25641987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f256259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f25641987f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35526/0 (socket says 172.21.15.93:35526) 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f256259c700 1 -- 172.21.15.93:0/561482855 learned_addr learned my addr 172.21.15.93:0/561482855 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f256259c700 1 -- 172.21.15.93:0/561482855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2554009550 con 0x7f2564104e20 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f256259c700 1 --2- 172.21.15.93:0/561482855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f25641987f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2554006af0 tx=0x7f2554011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f255b7fe700 1 -- 172.21.15.93:0/561482855 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f2554003a40 con 0x7f2564104e20 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f255b7fe700 1 -- 172.21.15.93:0/561482855 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f2554003ba0 con 0x7f2564104e20 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f2568e6b700 1 -- 172.21.15.93:0/561482855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2564108d70 con 0x7f2564104e20 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f255b7fe700 1 -- 172.21.15.93:0/561482855 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f255401a6e0 con 0x7f2564104e20 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f2568e6b700 1 -- 172.21.15.93:0/561482855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2564108f00 con 0x7f2564104e20 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f255b7fe700 1 -- 172.21.15.93:0/561482855 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f255401ac50 con 0x7f2564104e20 2023-12-14T18:18:49.618 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.193+0000 7f255b7fe700 1 -- 172.21.15.93:0/561482855 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f2554019700 con 0x7f2564104e20 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.194+0000 7f2568e6b700 1 -- 172.21.15.93:0/561482855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25641069b0 con 0x7f2564104e20 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.196+0000 7f255b7fe700 1 -- 172.21.15.93:0/561482855 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f25641069b0 con 0x7f2564104e20 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.263+0000 7f2568e6b700 1 -- 172.21.15.93:0/561482855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7f25641992d0 con 0x7f2564104e20 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.263+0000 7f255b7fe700 1 -- 172.21.15.93:0/561482855 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+175 (secure 0 0 0) 0x7f25641992d0 con 0x7f2564104e20 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.264+0000 7f2568e6b700 1 -- 172.21.15.93:0/561482855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 msgr2=0x7f25641987f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.264+0000 7f2568e6b700 1 --2- 172.21.15.93:0/561482855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f25641987f0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f2554006af0 tx=0x7f2554011ad0 comp rx=0 tx=0).stop 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.264+0000 7f2568e6b700 1 -- 172.21.15.93:0/561482855 shutdown_connections 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.264+0000 7f2568e6b700 1 --2- 172.21.15.93:0/561482855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2564104e20 0x7f25641987f0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.264+0000 7f2568e6b700 1 -- 172.21.15.93:0/561482855 >> 172.21.15.93:0/561482855 conn(0x7f25640ff650 msgr2=0x7f2564100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.264+0000 7f2568e6b700 1 -- 172.21.15.93:0/561482855 shutdown_connections 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:49.264+0000 7f2568e6b700 1 -- 172.21.15.93:0/561482855 wait complete. 2023-12-14T18:18:49.619 INFO:teuthology.orchestra.run.smithi093.stdout:Restarting the monitor... 2023-12-14T18:18:49.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:49 smithi093 systemd[1]: Stopping Ceph mon.a for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:18:49.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:49 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-a[41561]: 2023-12-14T18:18:49.812+0000 7f045912d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:18:49.944 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:49 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-a[41561]: 2023-12-14T18:18:49.812+0000 7f045912d700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2023-12-14T18:18:50.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:49 smithi093 bash[41911]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-a 2023-12-14T18:18:50.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 systemd[1]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.a.service: Succeeded. 2023-12-14T18:18:50.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 systemd[1]: Stopped Ceph mon.a for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:18:50.605 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 systemd[1]: Starting Ceph mon.a for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:18:50.662 INFO:teuthology.orchestra.run.smithi093.stdout:Setting mon public_network to 172.21.0.0/20 2023-12-14T18:18:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: set uid:gid to 167:167 (ceph:ceph) 2023-12-14T18:18:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev), process ceph-mon, pid 7 2023-12-14T18:18:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: pidfile_write: ignore empty --pid-file 2023-12-14T18:18:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: load: jerasure load: lrc 2023-12-14T18:18:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: RocksDB version: 7.9.2 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Git sha 0 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Compile date 2023-12-13 16:19:48 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: DB SUMMARY 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: DB Session ID: DE682YF3FWEEU200YI1N 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: CURRENT file: CURRENT 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 bash[42032]: 7331f67f21bf190aebe45b8e292bec86091bbb052edbb33c4541eadd429a0984 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: IDENTITY file: IDENTITY 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 97348 ; 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.error_if_exists: 0 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.create_if_missing: 0 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.paranoid_checks: 1 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-14T18:18:51.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.env: 0x55c69de58720 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.fs: PosixFileSystem 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.info_log: 0x55c6a0fc93c0 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_file_opening_threads: 16 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.statistics: (nil) 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.use_fsync: 0 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_log_file_size: 0 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.keep_log_file_num: 1000 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.recycle_log_file_num: 0 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.allow_fallocate: 1 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.allow_mmap_reads: 0 2023-12-14T18:18:51.067 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.allow_mmap_writes: 0 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.use_direct_reads: 0 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.create_missing_column_families: 0 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.db_log_dir: 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.wal_dir: 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.advise_random_on_open: 1 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.db_write_buffer_size: 0 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.write_buffer_manager: 0x55c6a023a3c0 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-14T18:18:51.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.rate_limiter: (nil) 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.wal_recovery_mode: 2 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.enable_thread_tracking: 0 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.enable_pipelined_write: 0 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.unordered_write: 0 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.row_cache: None 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.wal_filter: None 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.allow_ingest_behind: 0 2023-12-14T18:18:51.069 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.two_write_queues: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.manual_wal_flush: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.wal_compression: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.atomic_flush: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.log_readahead_size: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.best_efforts_recovery: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.allow_data_in_errors: 0 2023-12-14T18:18:51.070 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.db_host_id: __hostname__ 2023-12-14T18:18:51.071 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-14T18:18:51.071 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_background_jobs: 2 2023-12-14T18:18:51.071 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_background_compactions: -1 2023-12-14T18:18:51.071 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_subcompactions: 1 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_total_wal_size: 0 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_open_files: -1 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bytes_per_sync: 0 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_readahead_size: 0 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_background_flushes: -1 2023-12-14T18:18:51.072 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Compression algorithms supported: 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: kZSTD supported: 0 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: kXpressCompression supported: 0 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: kLZ4HCCompression supported: 1 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: kZlibCompression supported: 1 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: kSnappyCompression supported: 1 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: kLZ4Compression supported: 1 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: kBZip2Compression supported: 0 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.merge_operator: 2023-12-14T18:18:51.073 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_filter: None 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_filter_factory: None 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.sst_partitioner_factory: None 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55c6a0fc94e0) 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: cache_index_and_filter_blocks: 1 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: pin_top_level_index_and_filter: 1 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: index_type: 0 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: data_block_index_type: 0 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: index_shortening: 1 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: checksum: 4 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: no_block_cache: 0 2023-12-14T18:18:51.074 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache: 0x55c6a02db1f0 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache_name: BinnedLRUCache 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache_options: 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: capacity : 536870912 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: num_shard_bits : 4 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: strict_capacity_limit : 0 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: high_pri_pool_ratio: 0.000 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_cache_compressed: (nil) 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: persistent_cache: (nil) 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_size: 4096 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_size_deviation: 10 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_restart_interval: 16 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: index_block_restart_interval: 1 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: metadata_block_size: 4096 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: partition_filters: 0 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: use_delta_encoding: 1 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: filter_policy: bloomfilter 2023-12-14T18:18:51.075 INFO:journalctl@ceph.mon.a.smithi093.stdout: whole_key_filtering: 1 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout: verify_compression: 0 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout: read_amp_bytes_per_bit: 0 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout: format_version: 5 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout: enable_index_compression: 1 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout: block_align: 0 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout: max_auto_readahead_size: 262144 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout: prepopulate_block_cache: 0 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout: initial_auto_readahead_size: 8192 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout: num_file_reads_for_auto_readahead: 2 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.write_buffer_size: 33554432 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_write_buffer_number: 2 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression: NoCompression 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression: Disabled 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.prefix_extractor: nullptr 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-14T18:18:51.076 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.num_levels: 7 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression_opts.level: 32767 2023-12-14T18:18:51.077 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression_opts.strategy: 0 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression_opts.enabled: false 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.target_file_size_base: 67108864 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-14T18:18:51.078 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.arena_block_size: 1048576 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.disable_auto_compactions: 0 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-14T18:18:51.079 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.table_properties_collectors: 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.inplace_update_support: 0 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.bloom_locality: 0 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.max_successive_merges: 0 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-14T18:18:51.080 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.paranoid_file_checks: 0 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.force_consistency_checks: 1 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.report_bg_io_stats: 0 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.ttl: 2592000 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.enable_blob_files: false 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.min_blob_size: 0 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.blob_file_size: 268435456 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-14T18:18:51.081 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.blob_file_starting_level: 0 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 0ec43d55-1b71-41be-a6ed-bba2500da597 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702577930659575, "job": 1, "event": "recovery_started", "wal_files": [9]} 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702577930660587, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 92765, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 307, "table_properties": {"data_size": 90764, "index_size": 283, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 709, "raw_key_size": 13602, "raw_average_key_size": 50, "raw_value_size": 83344, "raw_average_value_size": 308, "num_data_blocks": 11, "num_entries": 270, "num_filter_entries": 270, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702577930, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "0ec43d55-1b71-41be-a6ed-bba2500da597", "db_session_id": "DE682YF3FWEEU200YI1N", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702577930660656, "job": 1, "event": "recovery_finished"} 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 systemd[1]: Started Ceph mon.a for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55c6a0378000 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: DB pointer 0x55c6a0364000 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-14T18:18:51.082 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** DB Stats ** 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** Compaction Stats [default] ** 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: L0 2/0 92.43 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:18:51.083 INFO:journalctl@ceph.mon.a.smithi093.stdout: Sum 2/0 92.43 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 136.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** Compaction Stats [default] ** 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 136.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-14T18:18:51.084 INFO:journalctl@ceph.mon.a.smithi093.stdout: Cumulative compaction: 0.00 GB write, 27.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: Interval compaction: 0.00 GB write, 27.07 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: Block cache BinnedLRUCache@0x55c6a02db1f0#7 capacity: 512.00 MB usage: 1.36 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 1.2e-05 secs_since: 0 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.89 KB,0.000169873%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: starting mon.a rank 0 at public addrs [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] at bind addrs [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mon.a@-1(???) e1 preinit fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mon.a@-1(???).mds e1 new map 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mon.a@-1(???).mds e1 print_map 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: e1 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:18:51.085 INFO:journalctl@ceph.mon.a.smithi093.stdout: legacy client fscid: -1 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout: 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout: No filesystems configured 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta expand map: {default=false} 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta from 'false' to 'false' 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta expanded map: {default=false} 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta expand map: {default=info} 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta from 'info' to 'info' 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta expanded map: {default=info} 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta expand map: {default=daemon} 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta from 'daemon' to 'daemon' 2023-12-14T18:18:51.086 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta expanded map: {default=daemon} 2023-12-14T18:18:51.087 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta expand map: {default=debug} 2023-12-14T18:18:51.087 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta from 'debug' to 'debug' 2023-12-14T18:18:51.087 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: expand_channel_meta expanded map: {default=debug} 2023-12-14T18:18:51.087 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mon.a is new leader, mons a in quorum (ranks 0) 2023-12-14T18:18:51.087 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: monmap e1: 1 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0]} removed_ranks: {} 2023-12-14T18:18:51.087 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: fsmap 2023-12-14T18:18:51.087 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: osdmap e1: 0 total, 0 up, 0 in 2023-12-14T18:18:51.087 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:50 smithi093 ceph-mon[42070]: mgrmap e1: no daemons active 2023-12-14T18:18:51.508 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.047+0000 7f176a1d2700 1 Processor -- start 2023-12-14T18:18:51.508 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.047+0000 7f176a1d2700 1 -- start start 2023-12-14T18:18:51.508 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.047+0000 7f176a1d2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:51.508 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.047+0000 7f176a1d2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764079e00 con 0x7f17641064f0 2023-12-14T18:18:51.508 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.047+0000 7f17637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:51.508 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.047+0000 7f17637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35542/0 (socket says 172.21.15.93:35542) 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.047+0000 7f17637fe700 1 -- 172.21.15.93:0/973681080 learned_addr learned my addr 172.21.15.93:0/973681080 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.048+0000 7f17637fe700 1 -- 172.21.15.93:0/973681080 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1764108e10 con 0x7f17641064f0 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.048+0000 7f17637fe700 1 --2- 172.21.15.93:0/973681080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641088d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17540098a0 tx=0x7f1754009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84f66b148808c55b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.048+0000 7f17627fc700 1 -- 172.21.15.93:0/973681080 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1754004160 con 0x7f17641064f0 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.048+0000 7f17627fc700 1 -- 172.21.15.93:0/973681080 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f17540042c0 con 0x7f17641064f0 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.048+0000 7f17627fc700 1 -- 172.21.15.93:0/973681080 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1754011a00 con 0x7f17641064f0 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.048+0000 7f176a1d2700 1 -- 172.21.15.93:0/973681080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 msgr2=0x7f17641088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.048+0000 7f176a1d2700 1 --2- 172.21.15.93:0/973681080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641088d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f17540098a0 tx=0x7f1754009bb0 comp rx=0 tx=0).stop 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.049+0000 7f176a1d2700 1 -- 172.21.15.93:0/973681080 shutdown_connections 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.049+0000 7f176a1d2700 1 --2- 172.21.15.93:0/973681080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641088d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.049+0000 7f176a1d2700 1 -- 172.21.15.93:0/973681080 >> 172.21.15.93:0/973681080 conn(0x7f17640ffed0 msgr2=0x7f1764102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.049+0000 7f176a1d2700 1 -- 172.21.15.93:0/973681080 shutdown_connections 2023-12-14T18:18:51.509 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.049+0000 7f176a1d2700 1 -- 172.21.15.93:0/973681080 wait complete. 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.049+0000 7f176a1d2700 1 Processor -- start 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.049+0000 7f176a1d2700 1 -- start start 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f176a1d2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f176a1d2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1764199cf0 con 0x7f17641064f0 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f17637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f17637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641997b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35544/0 (socket says 172.21.15.93:35544) 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f17637fe700 1 -- 172.21.15.93:0/2792812830 learned_addr learned my addr 172.21.15.93:0/2792812830 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f17637fe700 1 -- 172.21.15.93:0/2792812830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1754009550 con 0x7f17641064f0 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f17637fe700 1 --2- 172.21.15.93:0/2792812830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641997b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1754009fb0 tx=0x7f1754011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f1760ff9700 1 -- 172.21.15.93:0/2792812830 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1754003930 con 0x7f17641064f0 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f1760ff9700 1 -- 172.21.15.93:0/2792812830 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f1754003a90 con 0x7f17641064f0 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f176a1d2700 1 -- 172.21.15.93:0/2792812830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1764109d40 con 0x7f17641064f0 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f1760ff9700 1 -- 172.21.15.93:0/2792812830 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1754019610 con 0x7f17641064f0 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.050+0000 7f176a1d2700 1 -- 172.21.15.93:0/2792812830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1764109bb0 con 0x7f17641064f0 2023-12-14T18:18:51.510 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.051+0000 7f1760ff9700 1 -- 172.21.15.93:0/2792812830 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1754019bd0 con 0x7f17641064f0 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.051+0000 7f1760ff9700 1 -- 172.21.15.93:0/2792812830 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f17540243f0 con 0x7f17641064f0 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.051+0000 7f176a1d2700 1 -- 172.21.15.93:0/2792812830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1764109ed0 con 0x7f17641064f0 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.053+0000 7f1760ff9700 1 -- 172.21.15.93:0/2792812830 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7f1754003c00 con 0x7f17641064f0 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.121+0000 7f176a1d2700 1 -- 172.21.15.93:0/2792812830 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7f1764062120 con 0x7f17641064f0 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.126+0000 7f1760ff9700 1 -- 172.21.15.93:0/2792812830 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7f175401a9d0 con 0x7f17641064f0 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.126+0000 7f176a1d2700 1 -- 172.21.15.93:0/2792812830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 msgr2=0x7f17641997b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.126+0000 7f176a1d2700 1 --2- 172.21.15.93:0/2792812830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641997b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f1754009fb0 tx=0x7f1754011ad0 comp rx=0 tx=0).stop 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.127+0000 7f176a1d2700 1 -- 172.21.15.93:0/2792812830 shutdown_connections 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.127+0000 7f176a1d2700 1 --2- 172.21.15.93:0/2792812830 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17641064f0 0x7f17641997b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.127+0000 7f176a1d2700 1 -- 172.21.15.93:0/2792812830 >> 172.21.15.93:0/2792812830 conn(0x7f17640ffed0 msgr2=0x7f17641015c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.127+0000 7f176a1d2700 1 -- 172.21.15.93:0/2792812830 shutdown_connections 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:51.127+0000 7f176a1d2700 1 -- 172.21.15.93:0/2792812830 wait complete. 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:Wrote config to /etc/ceph/ceph.conf 2023-12-14T18:18:51.511 INFO:teuthology.orchestra.run.smithi093.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:18:51.512 INFO:teuthology.orchestra.run.smithi093.stdout:Creating mgr... 2023-12-14T18:18:51.512 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying port 9283 ... 2023-12-14T18:18:51.512 INFO:teuthology.orchestra.run.smithi093.stdout:Verifying port 8765 ... 2023-12-14T18:18:51.630 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.x 2023-12-14T18:18:51.630 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Failed to reset failed state of unit ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.x.service: Unit ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.x.service not loaded. 2023-12-14T18:18:51.719 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715.target.wants/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.x.service → /etc/systemd/system/ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@.service. 2023-12-14T18:18:52.003 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:51 smithi093 systemd[1]: Starting Ceph mgr.x for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:18:52.206 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-14T18:18:52.206 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-12-14T18:18:52.213 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-14T18:18:52.213 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-12-14T18:18:52.213 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-12-14T18:18:52.213 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to enable service . firewalld.service is not available 2023-12-14T18:18:52.221 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-14T18:18:52.221 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-12-14T18:18:52.229 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-14T18:18:52.230 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-12-14T18:18:52.230 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-12-14T18:18:52.230 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2023-12-14T18:18:52.230 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr to start... 2023-12-14T18:18:52.230 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr... 2023-12-14T18:18:52.400 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:52 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2792812830' entity='client.admin' 2023-12-14T18:18:52.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:52 smithi093 bash[42309]: 0e2a0b04bdb9eb9dfee268e81ca7e106b23eb18bc33fc3fd126f5797730ebef6 2023-12-14T18:18:52.400 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:52 smithi093 systemd[1]: Started Ceph mgr.x for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:18:52.789 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:52 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:52.498+0000 7f5bec2e1200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:18:52.789 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:52 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:52.607+0000 7f5bec2e1200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:18:52.789 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:52 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:52.699+0000 7f5bec2e1200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:18:52.790 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:52 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:52.787+0000 7f5bec2e1200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:18:53.044 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:18:53.047 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:18:53.047 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:18:53.047 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-12-14T18:18:53.047 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-14T18:18:53.047 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-14T18:18:53.047 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-14T18:18:53.047 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:18:53.048 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-14T18:18:53.049 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-14T18:18:53.049 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-14T18:18:53.055 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:53.056 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:18:53.057 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-12-14T18:18:47.086612+0000", 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.612+0000 7fc37e7aa700 1 Processor -- start 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.612+0000 7fc37e7aa700 1 -- start start 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.612+0000 7fc37e7aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc3781088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.613+0000 7fc37e7aa700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc378079e20 con 0x7fc3781064d0 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.613+0000 7fc377fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc3781088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.613+0000 7fc377fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc3781088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32780/0 (socket says 172.21.15.93:32780) 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.613+0000 7fc377fff700 1 -- 172.21.15.93:0/1378665386 learned_addr learned my addr 172.21.15.93:0/1378665386 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:53.058 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.613+0000 7fc377fff700 1 -- 172.21.15.93:0/1378665386 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc378108df0 con 0x7fc3781064d0 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.613+0000 7fc377fff700 1 --2- 172.21.15.93:0/1378665386 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc3781088b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3680098a0 tx=0x7fc368009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ffd0a893b89cab6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.613+0000 7fc376ffd700 1 -- 172.21.15.93:0/1378665386 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc368004160 con 0x7fc3781064d0 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.614+0000 7fc376ffd700 1 -- 172.21.15.93:0/1378665386 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc3680042c0 con 0x7fc3781064d0 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.614+0000 7fc376ffd700 1 -- 172.21.15.93:0/1378665386 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc368011a00 con 0x7fc3781064d0 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.614+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1378665386 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 msgr2=0x7fc3781088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.614+0000 7fc37e7aa700 1 --2- 172.21.15.93:0/1378665386 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc3781088b0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3680098a0 tx=0x7fc368009bb0 comp rx=0 tx=0).stop 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.614+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1378665386 shutdown_connections 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.614+0000 7fc37e7aa700 1 --2- 172.21.15.93:0/1378665386 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc3781088b0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.614+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1378665386 >> 172.21.15.93:0/1378665386 conn(0x7fc3780ffed0 msgr2=0x7fc378102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.614+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1378665386 shutdown_connections 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.614+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1378665386 wait complete. 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.615+0000 7fc37e7aa700 1 Processor -- start 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.615+0000 7fc37e7aa700 1 -- start start 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.615+0000 7fc37e7aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc378199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:53.059 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.615+0000 7fc37e7aa700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc378199cc0 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.615+0000 7fc377fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc378199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.615+0000 7fc377fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc378199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32790/0 (socket says 172.21.15.93:32790) 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.615+0000 7fc377fff700 1 -- 172.21.15.93:0/1404535745 learned_addr learned my addr 172.21.15.93:0/1404535745 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.616+0000 7fc377fff700 1 -- 172.21.15.93:0/1404535745 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc368009550 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.616+0000 7fc377fff700 1 --2- 172.21.15.93:0/1404535745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc378199780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc368009fb0 tx=0x7fc368011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.616+0000 7fc3757fa700 1 -- 172.21.15.93:0/1404535745 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc368003930 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.616+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1404535745 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc378109d20 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.616+0000 7fc3757fa700 1 -- 172.21.15.93:0/1404535745 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc368003a90 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.616+0000 7fc3757fa700 1 -- 172.21.15.93:0/1404535745 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc368019610 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.616+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1404535745 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc378109b90 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.616+0000 7fc3757fa700 1 -- 172.21.15.93:0/1404535745 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fc368019bf0 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.616+0000 7fc3757fa700 1 -- 172.21.15.93:0/1404535745 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc3680243f0 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.617+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1404535745 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc378062120 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.619+0000 7fc3757fa700 1 -- 172.21.15.93:0/1404535745 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fc368003c00 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.691+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1404535745 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc37819a200 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.692+0000 7fc3757fa700 1 -- 172.21.15.93:0/1404535745 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc36801a9d0 con 0x7fc3781064d0 2023-12-14T18:18:53.060 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.693+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1404535745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 msgr2=0x7fc378199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:53.061 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.693+0000 7fc37e7aa700 1 --2- 172.21.15.93:0/1404535745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc378199780 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc368009fb0 tx=0x7fc368011ad0 comp rx=0 tx=0).stop 2023-12-14T18:18:53.061 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.693+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1404535745 shutdown_connections 2023-12-14T18:18:53.061 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.693+0000 7fc37e7aa700 1 --2- 172.21.15.93:0/1404535745 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3781064d0 0x7fc378199780 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:53.061 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.693+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1404535745 >> 172.21.15.93:0/1404535745 conn(0x7fc3780ffed0 msgr2=0x7fc378101570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:53.061 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.693+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1404535745 shutdown_connections 2023-12-14T18:18:53.061 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:52.693+0000 7fc37e7aa700 1 -- 172.21.15.93:0/1404535745 wait complete. 2023-12-14T18:18:53.061 INFO:teuthology.orchestra.run.smithi093.stdout:mgr not available, waiting (1/15)... 2023-12-14T18:18:53.413 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:53 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1404535745' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-14T18:18:53.727 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:53.411+0000 7f5bec2e1200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:18:53.727 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:53.641+0000 7f5bec2e1200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:18:54.064 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:53.726+0000 7f5bec2e1200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:18:54.064 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:53.834+0000 7f5bec2e1200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:18:54.064 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:53.923+0000 7f5bec2e1200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:18:54.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:54.115+0000 7f5bec2e1200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:18:54.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:54.198+0000 7f5bec2e1200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:18:54.828 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:54.567+0000 7f5bec2e1200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:18:55.110 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:54.826+0000 7f5bec2e1200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:18:55.110 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:55 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:55.108+0000 7f5bec2e1200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:18:55.554 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:55 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:55.209+0000 7f5bec2e1200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:18:55.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:55 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2599626665' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-12-14T18:18:55.913 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 4, 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-14T18:18:55.914 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-14T18:18:55.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-14T18:18:55.921 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-14T18:18:55.922 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-14T18:18:55.923 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-12-14T18:18:47.086612+0000", 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.445+0000 7fba8d5e9700 1 Processor -- start 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.445+0000 7fba8d5e9700 1 -- start start 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.445+0000 7fba8d5e9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.445+0000 7fba8d5e9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba88106410 con 0x7fba88105ac0 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.445+0000 7fba86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.446+0000 7fba86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88105ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32796/0 (socket says 172.21.15.93:32796) 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.446+0000 7fba86ffd700 1 -- 172.21.15.93:0/933998052 learned_addr learned my addr 172.21.15.93:0/933998052 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.446+0000 7fba86ffd700 1 -- 172.21.15.93:0/933998052 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba88106550 con 0x7fba88105ac0 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.446+0000 7fba86ffd700 1 --2- 172.21.15.93:0/933998052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88105ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba780098a0 tx=0x7fba78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98ba0f1b5b85d8a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:55.924 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.446+0000 7fba85ffb700 1 -- 172.21.15.93:0/933998052 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba78004160 con 0x7fba88105ac0 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.446+0000 7fba85ffb700 1 -- 172.21.15.93:0/933998052 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba780042c0 con 0x7fba88105ac0 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.446+0000 7fba85ffb700 1 -- 172.21.15.93:0/933998052 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba78011a00 con 0x7fba88105ac0 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.447+0000 7fba8d5e9700 1 -- 172.21.15.93:0/933998052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 msgr2=0x7fba88105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.447+0000 7fba8d5e9700 1 --2- 172.21.15.93:0/933998052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88105ed0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fba780098a0 tx=0x7fba78009bb0 comp rx=0 tx=0).stop 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.447+0000 7fba8d5e9700 1 -- 172.21.15.93:0/933998052 shutdown_connections 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.447+0000 7fba8d5e9700 1 --2- 172.21.15.93:0/933998052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88105ed0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.447+0000 7fba8d5e9700 1 -- 172.21.15.93:0/933998052 >> 172.21.15.93:0/933998052 conn(0x7fba88101170 msgr2=0x7fba88103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.447+0000 7fba8d5e9700 1 -- 172.21.15.93:0/933998052 shutdown_connections 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.447+0000 7fba8d5e9700 1 -- 172.21.15.93:0/933998052 wait complete. 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.448+0000 7fba8d5e9700 1 Processor -- start 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.448+0000 7fba8d5e9700 1 -- start start 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.448+0000 7fba8d5e9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.448+0000 7fba8d5e9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba88199d60 con 0x7fba88105ac0 2023-12-14T18:18:55.925 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.448+0000 7fba86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.448+0000 7fba86ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32812/0 (socket says 172.21.15.93:32812) 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.448+0000 7fba86ffd700 1 -- 172.21.15.93:0/2599626665 learned_addr learned my addr 172.21.15.93:0/2599626665 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.449+0000 7fba86ffd700 1 -- 172.21.15.93:0/2599626665 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba78009550 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.449+0000 7fba86ffd700 1 --2- 172.21.15.93:0/2599626665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88199820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba78009fb0 tx=0x7fba78011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.449+0000 7fba6ffff700 1 -- 172.21.15.93:0/2599626665 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba78003930 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.449+0000 7fba6ffff700 1 -- 172.21.15.93:0/2599626665 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fba78003a90 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.449+0000 7fba8d5e9700 1 -- 172.21.15.93:0/2599626665 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba88107b10 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.449+0000 7fba6ffff700 1 -- 172.21.15.93:0/2599626665 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fba78019610 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.449+0000 7fba8d5e9700 1 -- 172.21.15.93:0/2599626665 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba88107980 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.450+0000 7fba6ffff700 1 -- 172.21.15.93:0/2599626665 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fba780198f0 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.450+0000 7fba6ffff700 1 -- 172.21.15.93:0/2599626665 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fba7802da60 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.450+0000 7fba8d5e9700 1 -- 172.21.15.93:0/2599626665 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba88109f90 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.452+0000 7fba6ffff700 1 -- 172.21.15.93:0/2599626665 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fba78003c00 con 0x7fba88105ac0 2023-12-14T18:18:55.926 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.524+0000 7fba8d5e9700 1 -- 172.21.15.93:0/2599626665 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fba8819a2a0 con 0x7fba88105ac0 2023-12-14T18:18:55.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.525+0000 7fba6ffff700 1 -- 172.21.15.93:0/2599626665 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fba78019d30 con 0x7fba88105ac0 2023-12-14T18:18:55.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.526+0000 7fba8d5e9700 1 -- 172.21.15.93:0/2599626665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 msgr2=0x7fba88199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:18:55.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.526+0000 7fba8d5e9700 1 --2- 172.21.15.93:0/2599626665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88199820 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fba78009fb0 tx=0x7fba78011ad0 comp rx=0 tx=0).stop 2023-12-14T18:18:55.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.526+0000 7fba8d5e9700 1 -- 172.21.15.93:0/2599626665 shutdown_connections 2023-12-14T18:18:55.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.526+0000 7fba8d5e9700 1 --2- 172.21.15.93:0/2599626665 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba88105ac0 0x7fba88199820 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:18:55.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.526+0000 7fba8d5e9700 1 -- 172.21.15.93:0/2599626665 >> 172.21.15.93:0/2599626665 conn(0x7fba88101170 msgr2=0x7fba88101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:18:55.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.526+0000 7fba8d5e9700 1 -- 172.21.15.93:0/2599626665 shutdown_connections 2023-12-14T18:18:55.927 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:55.526+0000 7fba8d5e9700 1 -- 172.21.15.93:0/2599626665 wait complete. 2023-12-14T18:18:55.927 INFO:teuthology.orchestra.run.smithi093.stdout:mgr not available, waiting (2/15)... 2023-12-14T18:18:56.233 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:55 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:55.991+0000 7f5bec2e1200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:18:56.233 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:56.086+0000 7f5bec2e1200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:18:56.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:56.231+0000 7f5bec2e1200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:18:57.226 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:56.851+0000 7f5bec2e1200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:18:57.226 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:57 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:57.032+0000 7f5bec2e1200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:18:57.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:57 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:57.224+0000 7f5bec2e1200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:18:58.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:57 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:57.933+0000 7f5bec2e1200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:18:58.695 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:58.412+0000 7f5bec2e1200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:18:58.696 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:58.503+0000 7f5bec2e1200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:18:58.696 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:18:58.694+0000 7f5bec2e1200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:18:58.962 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: Activating manager daemon x 2023-12-14T18:18:58.962 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: mgrmap e2: x(active, starting, since 0.002991s) 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: Manager daemon x is now available 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' 2023-12-14T18:18:58.963 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:58 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' 2023-12-14T18:19:00.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:59 smithi093 ceph-mon[42070]: from='mgr.14100 172.21.15.93:0/3545070831' entity='mgr.x' 2023-12-14T18:19:00.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:59 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3859145732' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-14T18:19:00.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:18:59 smithi093 ceph-mon[42070]: mgrmap e3: x(active, since 1.00633s) 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-14T18:19:00.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-14T18:19:00.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-14T18:19:00.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-14T18:19:00.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-14T18:19:00.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-14T18:19:00.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:00.547 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-14T18:19:00.548 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": false, 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-12-14T18:18:47.086612+0000", 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-14T18:19:00.549 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.254+0000 7fc1f8613700 1 Processor -- start 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.254+0000 7fc1f8613700 1 -- start start 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f8613700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f8613700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0079e60 con 0x7fc1f0106490 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f63af700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f63af700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32860/0 (socket says 172.21.15.93:32860) 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f63af700 1 -- 172.21.15.93:0/2336165107 learned_addr learned my addr 172.21.15.93:0/2336165107 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f63af700 1 -- 172.21.15.93:0/2336165107 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1f0108db0 con 0x7fc1f0106490 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f63af700 1 --2- 172.21.15.93:0/2336165107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0108870 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40098a0 tx=0x7fc1e4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ce1dc450d01ac48 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f53ad700 1 -- 172.21.15.93:0/2336165107 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e4004160 con 0x7fc1f0106490 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f53ad700 1 -- 172.21.15.93:0/2336165107 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1e40042c0 con 0x7fc1f0106490 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.255+0000 7fc1f53ad700 1 -- 172.21.15.93:0/2336165107 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e4011a00 con 0x7fc1f0106490 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.256+0000 7fc1f8613700 1 -- 172.21.15.93:0/2336165107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 msgr2=0x7fc1f0108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.256+0000 7fc1f8613700 1 --2- 172.21.15.93:0/2336165107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0108870 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fc1e40098a0 tx=0x7fc1e4009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.256+0000 7fc1f8613700 1 -- 172.21.15.93:0/2336165107 shutdown_connections 2023-12-14T18:19:00.550 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.256+0000 7fc1f8613700 1 --2- 172.21.15.93:0/2336165107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0108870 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.256+0000 7fc1f8613700 1 -- 172.21.15.93:0/2336165107 >> 172.21.15.93:0/2336165107 conn(0x7fc1f00ffed0 msgr2=0x7fc1f0102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.256+0000 7fc1f8613700 1 -- 172.21.15.93:0/2336165107 shutdown_connections 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.256+0000 7fc1f8613700 1 -- 172.21.15.93:0/2336165107 wait complete. 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1f8613700 1 Processor -- start 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1f8613700 1 -- start start 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1f8613700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1f8613700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1f0106330 con 0x7fc1f0106490 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1f63af700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1f63af700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32866/0 (socket says 172.21.15.93:32866) 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1f63af700 1 -- 172.21.15.93:0/3859145732 learned_addr learned my addr 172.21.15.93:0/3859145732 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1f63af700 1 -- 172.21.15.93:0/3859145732 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e4009550 con 0x7fc1f0106490 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1f63af700 1 --2- 172.21.15.93:0/3859145732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0105df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4006af0 tx=0x7fc1e4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.257+0000 7fc1e37fe700 1 -- 172.21.15.93:0/3859145732 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e4003a40 con 0x7fc1f0106490 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.258+0000 7fc1e37fe700 1 -- 172.21.15.93:0/3859145732 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fc1e4003ba0 con 0x7fc1f0106490 2023-12-14T18:19:00.551 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.258+0000 7fc1e37fe700 1 -- 172.21.15.93:0/3859145732 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc1e401a6e0 con 0x7fc1f0106490 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.258+0000 7fc1f8613700 1 -- 172.21.15.93:0/3859145732 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1f0109d40 con 0x7fc1f0106490 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.258+0000 7fc1f8613700 1 -- 172.21.15.93:0/3859145732 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1f0109bb0 con 0x7fc1f0106490 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.258+0000 7fc1e37fe700 1 -- 172.21.15.93:0/3859145732 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 2) v1 ==== 45239+0+0 (secure 0 0 0) 0x7fc1e4024a60 con 0x7fc1f0106490 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.259+0000 7fc1e37fe700 1 -- 172.21.15.93:0/3859145732 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fc1e404c2b0 con 0x7fc1f0106490 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.259+0000 7fc1f8613700 1 -- 172.21.15.93:0/3859145732 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1f0062120 con 0x7fc1f0106490 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.261+0000 7fc1e37fe700 1 -- 172.21.15.93:0/3859145732 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72437 (secure 0 0 0) 0x7fc1e4024430 con 0x7fc1f0106490 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.331+0000 7fc1f8613700 1 -- 172.21.15.93:0/3859145732 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fc1f0104ba0 con 0x7fc1f0106490 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.331+0000 7fc1e37fe700 1 -- 172.21.15.93:0/3859145732 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fc1e4047090 con 0x7fc1f0106490 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.332+0000 7fc1f8613700 1 -- 172.21.15.93:0/3859145732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 msgr2=0x7fc1f0105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.332+0000 7fc1f8613700 1 --2- 172.21.15.93:0/3859145732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0105df0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fc1e4006af0 tx=0x7fc1e4011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.332+0000 7fc1f8613700 1 -- 172.21.15.93:0/3859145732 shutdown_connections 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.332+0000 7fc1f8613700 1 --2- 172.21.15.93:0/3859145732 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc1f0106490 0x7fc1f0105df0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.332+0000 7fc1f8613700 1 -- 172.21.15.93:0/3859145732 >> 172.21.15.93:0/3859145732 conn(0x7fc1f00ffed0 msgr2=0x7fc1f0100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.332+0000 7fc1f8613700 1 -- 172.21.15.93:0/3859145732 shutdown_connections 2023-12-14T18:19:00.552 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:18:59.332+0000 7fc1f8613700 1 -- 172.21.15.93:0/3859145732 wait complete. 2023-12-14T18:19:00.553 INFO:teuthology.orchestra.run.smithi093.stdout:mgr not available, waiting (3/15)... 2023-12-14T18:19:03.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:02 smithi093 ceph-mon[42070]: mgrmap e4: x(active, since 2s) 2023-12-14T18:19:04.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:04 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2122577146' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2023-12-14T18:19:06.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:06.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:19:06.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:19:06.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "health": { 2023-12-14T18:19:06.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2023-12-14T18:19:06.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "checks": {}, 2023-12-14T18:19:06.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mutes": [] 2023-12-14T18:19:06.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:06.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum": [ 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 0 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "a" 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "quorum_age": 13, 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "monmap": { 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "reef", 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_mons": 1 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osdmap": { 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2023-12-14T18:19:06.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgmap": { 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2023-12-14T18:19:06.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "fsmap": { 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "by_rank": [], 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "up:standby": 0 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap": { 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modules": [ 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "iostat", 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "nfs", 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "restful" 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ], 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:06.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "servicemap": { 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 1, 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "modified": "2023-12-14T18:18:47.086612+0000", 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "services": {} 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout }, 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "progress_events": {} 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.142+0000 7fb833adc700 1 Processor -- start 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.143+0000 7fb833adc700 1 -- start start 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.143+0000 7fb833adc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c1088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.143+0000 7fb833adc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c079e20 con 0x7fb82c1064d0 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.143+0000 7fb831878700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c1088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.143+0000 7fb831878700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c1088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33924/0 (socket says 172.21.15.93:33924) 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.143+0000 7fb831878700 1 -- 172.21.15.93:0/2354972705 learned_addr learned my addr 172.21.15.93:0/2354972705 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:06.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.143+0000 7fb831878700 1 -- 172.21.15.93:0/2354972705 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb82c108df0 con 0x7fb82c1064d0 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.143+0000 7fb831878700 1 --2- 172.21.15.93:0/2354972705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c1088b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8200098a0 tx=0x7fb820009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=287eb9768d89ec97 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.143+0000 7fb830876700 1 -- 172.21.15.93:0/2354972705 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb820004160 con 0x7fb82c1064d0 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.144+0000 7fb830876700 1 -- 172.21.15.93:0/2354972705 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb8200042c0 con 0x7fb82c1064d0 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.144+0000 7fb830876700 1 -- 172.21.15.93:0/2354972705 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb820011a00 con 0x7fb82c1064d0 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.144+0000 7fb833adc700 1 -- 172.21.15.93:0/2354972705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 msgr2=0x7fb82c1088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.144+0000 7fb833adc700 1 --2- 172.21.15.93:0/2354972705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c1088b0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fb8200098a0 tx=0x7fb820009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.144+0000 7fb833adc700 1 -- 172.21.15.93:0/2354972705 shutdown_connections 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.144+0000 7fb833adc700 1 --2- 172.21.15.93:0/2354972705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c1088b0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.144+0000 7fb833adc700 1 -- 172.21.15.93:0/2354972705 >> 172.21.15.93:0/2354972705 conn(0x7fb82c0ffed0 msgr2=0x7fb82c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.144+0000 7fb833adc700 1 -- 172.21.15.93:0/2354972705 shutdown_connections 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.144+0000 7fb833adc700 1 -- 172.21.15.93:0/2354972705 wait complete. 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.145+0000 7fb833adc700 1 Processor -- start 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.145+0000 7fb833adc700 1 -- start start 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.145+0000 7fb833adc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:06.500 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.145+0000 7fb833adc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb82c199cc0 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.145+0000 7fb831878700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.145+0000 7fb831878700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33936/0 (socket says 172.21.15.93:33936) 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.145+0000 7fb831878700 1 -- 172.21.15.93:0/2122577146 learned_addr learned my addr 172.21.15.93:0/2122577146 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.145+0000 7fb831878700 1 -- 172.21.15.93:0/2122577146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb820009550 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.145+0000 7fb831878700 1 --2- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c199780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb820009fb0 tx=0x7fb820011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.146+0000 7fb81effd700 1 -- 172.21.15.93:0/2122577146 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb820003930 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.146+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb82c109d20 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.146+0000 7fb81effd700 1 -- 172.21.15.93:0/2122577146 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fb820003a90 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.146+0000 7fb81effd700 1 -- 172.21.15.93:0/2122577146 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fb820019610 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.146+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb82c109b90 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.146+0000 7fb81effd700 1 -- 172.21.15.93:0/2122577146 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fb8200198d0 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.146+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb82c062120 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.147+0000 7fb81effd700 1 --2- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fb818038670 0x7fb81803ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.147+0000 7fb81effd700 1 -- 172.21.15.93:0/2122577146 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fb82004c470 con 0x7fb82c1064d0 2023-12-14T18:19:06.501 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.147+0000 7fb831077700 1 --2- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fb818038670 0x7fb81803ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.147+0000 7fb831077700 1 --2- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fb818038670 0x7fb81803ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.151+0000 7fb81effd700 1 -- 172.21.15.93:0/2122577146 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb820019b80 con 0x7fb82c1064d0 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.416+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fb82c19a450 con 0x7fb82c1064d0 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.416+0000 7fb81effd700 1 -- 172.21.15.93:0/2122577146 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1238 (secure 0 0 0) 0x7fb820018350 con 0x7fb82c1064d0 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.418+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fb818038670 msgr2=0x7fb81803ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.418+0000 7fb833adc700 1 --2- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fb818038670 0x7fb81803ab20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fb828006fd0 tx=0x7fb828006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.418+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 msgr2=0x7fb82c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.418+0000 7fb833adc700 1 --2- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c199780 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fb820009fb0 tx=0x7fb820011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.418+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 shutdown_connections 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.418+0000 7fb833adc700 1 --2- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fb818038670 0x7fb81803ab20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.418+0000 7fb833adc700 1 --2- 172.21.15.93:0/2122577146 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb82c1064d0 0x7fb82c199780 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.418+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 >> 172.21.15.93:0/2122577146 conn(0x7fb82c0ffed0 msgr2=0x7fb82c101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.419+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 shutdown_connections 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:04.419+0000 7fb833adc700 1 -- 172.21.15.93:0/2122577146 wait complete. 2023-12-14T18:19:06.502 INFO:teuthology.orchestra.run.smithi093.stdout:mgr is available 2023-12-14T18:19:07.703 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:07 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2512511253' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2023-12-14T18:19:07.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:07.750 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [global] 2023-12-14T18:19:07.750 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout fsid = 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:07.750 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2023-12-14T18:19:07.750 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2023-12-14T18:19:07.750 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2023-12-14T18:19:07.750 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2023-12-14T18:19:07.750 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:07.750 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [mgr] 2023-12-14T18:19:07.750 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout [osd] 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.191+0000 7f35b17ae700 1 Processor -- start 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.191+0000 7f35b17ae700 1 -- start start 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.191+0000 7f35b17ae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac104710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.191+0000 7f35b17ae700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac079de0 con 0x7f35ac104300 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.191+0000 7f35aaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.191+0000 7f35aaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33942/0 (socket says 172.21.15.93:33942) 2023-12-14T18:19:07.751 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.191+0000 7f35aaffd700 1 -- 172.21.15.93:0/2569833339 learned_addr learned my addr 172.21.15.93:0/2569833339 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.191+0000 7f35aaffd700 1 -- 172.21.15.93:0/2569833339 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35ac104c50 con 0x7f35ac104300 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35aaffd700 1 --2- 172.21.15.93:0/2569833339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac104710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f359c0098a0 tx=0x7f359c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b52e81bdabd881f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35a9ffb700 1 -- 172.21.15.93:0/2569833339 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f359c004160 con 0x7f35ac104300 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35a9ffb700 1 -- 172.21.15.93:0/2569833339 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f359c0042c0 con 0x7f35ac104300 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35a9ffb700 1 -- 172.21.15.93:0/2569833339 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f359c011a00 con 0x7f35ac104300 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35b17ae700 1 -- 172.21.15.93:0/2569833339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 msgr2=0x7f35ac104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35b17ae700 1 --2- 172.21.15.93:0/2569833339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac104710 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f359c0098a0 tx=0x7f359c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35b17ae700 1 -- 172.21.15.93:0/2569833339 shutdown_connections 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35b17ae700 1 --2- 172.21.15.93:0/2569833339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac104710 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35b17ae700 1 -- 172.21.15.93:0/2569833339 >> 172.21.15.93:0/2569833339 conn(0x7f35ac0ffed0 msgr2=0x7f35ac102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35b17ae700 1 -- 172.21.15.93:0/2569833339 shutdown_connections 2023-12-14T18:19:07.752 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.192+0000 7f35b17ae700 1 -- 172.21.15.93:0/2569833339 wait complete. 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.193+0000 7f35b17ae700 1 Processor -- start 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.193+0000 7f35b17ae700 1 -- start start 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.193+0000 7f35b17ae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.193+0000 7f35b17ae700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35ac19bee0 con 0x7f35ac104300 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.193+0000 7f35aaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.193+0000 7f35aaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33950/0 (socket says 172.21.15.93:33950) 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.193+0000 7f35aaffd700 1 -- 172.21.15.93:0/2512511253 learned_addr learned my addr 172.21.15.93:0/2512511253 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.194+0000 7f35aaffd700 1 -- 172.21.15.93:0/2512511253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c009550 con 0x7f35ac104300 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.194+0000 7f35aaffd700 1 --2- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac19b9a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f359c009fb0 tx=0x7f359c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.194+0000 7f3593fff700 1 -- 172.21.15.93:0/2512511253 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f359c003930 con 0x7f35ac104300 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.194+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35ac105c50 con 0x7f35ac104300 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.194+0000 7f3593fff700 1 -- 172.21.15.93:0/2512511253 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f359c003a90 con 0x7f35ac104300 2023-12-14T18:19:07.753 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.194+0000 7f3593fff700 1 -- 172.21.15.93:0/2512511253 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f359c019610 con 0x7f35ac104300 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.194+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35ac105ac0 con 0x7f35ac104300 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.195+0000 7f3593fff700 1 -- 172.21.15.93:0/2512511253 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7f359c0198d0 con 0x7f35ac104300 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.195+0000 7f3593fff700 1 --2- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f3594038670 0x7f359403ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.195+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35ac195250 con 0x7f35ac104300 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.195+0000 7f3593fff700 1 -- 172.21.15.93:0/2512511253 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f359c04c620 con 0x7f35ac104300 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.195+0000 7f35aa7fc700 1 --2- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f3594038670 0x7f359403ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.196+0000 7f35aa7fc700 1 --2- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f3594038670 0x7f359403ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35a0006fd0 tx=0x7f35a0006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.200+0000 7f3593fff700 1 -- 172.21.15.93:0/2512511253 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f359c019b80 con 0x7f35ac104300 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.384+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f35ac19c4d0 con 0x7f35ac104300 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.385+0000 7f3593fff700 1 -- 172.21.15.93:0/2512511253 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+373 (secure 0 0 0) 0x7f359c018350 con 0x7f35ac104300 2023-12-14T18:19:07.754 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f3594038670 msgr2=0x7f359403ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 --2- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f3594038670 0x7f359403ab20 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f35a0006fd0 tx=0x7f35a0006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 msgr2=0x7f35ac19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 --2- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac19b9a0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f359c009fb0 tx=0x7f359c011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 shutdown_connections 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 --2- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f3594038670 0x7f359403ab20 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 --2- 172.21.15.93:0/2512511253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f35ac104300 0x7f35ac19b9a0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 >> 172.21.15.93:0/2512511253 conn(0x7f35ac0ffed0 msgr2=0x7f35ac18fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 shutdown_connections 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:07.387+0000 7f35b17ae700 1 -- 172.21.15.93:0/2512511253 wait complete. 2023-12-14T18:19:07.755 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling cephadm module... 2023-12-14T18:19:08.744 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:08 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3254623753' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2023-12-14T18:19:08.745 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:08 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: ignoring --setuser ceph since I am not root 2023-12-14T18:19:08.745 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:08 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: ignoring --setgroup ceph since I am not root 2023-12-14T18:19:08.814 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.073+0000 7fe0b339e700 1 Processor -- start 2023-12-14T18:19:08.814 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.073+0000 7fe0b339e700 1 -- start start 2023-12-14T18:19:08.814 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.073+0000 7fe0b339e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac108810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:08.814 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.073+0000 7fe0b339e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac079cd0 con 0x7fe0ac106430 2023-12-14T18:19:08.814 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.073+0000 7fe0b239c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:08.814 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.073+0000 7fe0b239c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac108810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33954/0 (socket says 172.21.15.93:33954) 2023-12-14T18:19:08.814 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.073+0000 7fe0b239c700 1 -- 172.21.15.93:0/1868067206 learned_addr learned my addr 172.21.15.93:0/1868067206 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.074+0000 7fe0b239c700 1 -- 172.21.15.93:0/1868067206 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac079e10 con 0x7fe0ac106430 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.074+0000 7fe0b239c700 1 --2- 172.21.15.93:0/1868067206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac108810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80098a0 tx=0x7fe0a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89823e9265ad6671 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.074+0000 7fe0b139a700 1 -- 172.21.15.93:0/1868067206 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0a8004160 con 0x7fe0ac106430 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.074+0000 7fe0b139a700 1 -- 172.21.15.93:0/1868067206 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0a80042c0 con 0x7fe0ac106430 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.074+0000 7fe0b139a700 1 -- 172.21.15.93:0/1868067206 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0a8011a00 con 0x7fe0ac106430 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.074+0000 7fe0b339e700 1 -- 172.21.15.93:0/1868067206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 msgr2=0x7fe0ac108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.074+0000 7fe0b339e700 1 --2- 172.21.15.93:0/1868067206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac108810 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe0a80098a0 tx=0x7fe0a8009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.074+0000 7fe0b339e700 1 -- 172.21.15.93:0/1868067206 shutdown_connections 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.074+0000 7fe0b339e700 1 --2- 172.21.15.93:0/1868067206 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac108810 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.075+0000 7fe0b339e700 1 -- 172.21.15.93:0/1868067206 >> 172.21.15.93:0/1868067206 conn(0x7fe0ac0ffe30 msgr2=0x7fe0ac102240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.075+0000 7fe0b339e700 1 -- 172.21.15.93:0/1868067206 shutdown_connections 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.075+0000 7fe0b339e700 1 -- 172.21.15.93:0/1868067206 wait complete. 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.075+0000 7fe0b339e700 1 Processor -- start 2023-12-14T18:19:08.815 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.075+0000 7fe0b339e700 1 -- start start 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.075+0000 7fe0b339e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.075+0000 7fe0b339e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0ac199e70 con 0x7fe0ac106430 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0b239c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0b239c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33964/0 (socket says 172.21.15.93:33964) 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0b239c700 1 -- 172.21.15.93:0/3254623753 learned_addr learned my addr 172.21.15.93:0/3254623753 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0b239c700 1 -- 172.21.15.93:0/3254623753 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0a8009550 con 0x7fe0ac106430 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0b239c700 1 --2- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac199930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009fb0 tx=0x7fe0a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0a37fe700 1 -- 172.21.15.93:0/3254623753 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0a8003a40 con 0x7fe0ac106430 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0ac109b50 con 0x7fe0ac106430 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0a37fe700 1 -- 172.21.15.93:0/3254623753 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe0a8003ba0 con 0x7fe0ac106430 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0a37fe700 1 -- 172.21.15.93:0/3254623753 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe0a801a750 con 0x7fe0ac106430 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.076+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0ac1099c0 con 0x7fe0ac106430 2023-12-14T18:19:08.816 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.077+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0ac193630 con 0x7fe0ac106430 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.078+0000 7fe0a37fe700 1 -- 172.21.15.93:0/3254623753 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 4) v1 ==== 45472+0+0 (secure 0 0 0) 0x7fe0a8024b60 con 0x7fe0ac106430 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.078+0000 7fe0a37fe700 1 --2- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fe098038670 0x7fe09803ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.078+0000 7fe0a37fe700 1 -- 172.21.15.93:0/3254623753 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7fe0a804c6c0 con 0x7fe0ac106430 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.078+0000 7fe0b1b9b700 1 --2- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fe098038670 0x7fe09803ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.078+0000 7fe0b1b9b700 1 --2- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fe098038670 0x7fe09803ab20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe09c006fd0 tx=0x7fe09c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.082+0000 7fe0a37fe700 1 -- 172.21.15.93:0/3254623753 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe0a8018b40 con 0x7fe0ac106430 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.321+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7fe0ac074390 con 0x7fe0ac106430 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.442+0000 7fe0a37fe700 1 -- 172.21.15.93:0/3254623753 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe0a804b840 con 0x7fe0ac106430 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.444+0000 7fe0a37fe700 1 -- 172.21.15.93:0/3254623753 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7fe0a804bbc0 con 0x7fe0ac106430 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fe098038670 msgr2=0x7fe09803ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 --2- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fe098038670 0x7fe09803ab20 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe09c006fd0 tx=0x7fe09c006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:08.817 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 msgr2=0x7fe0ac199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:08.818 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 --2- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac199930 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe0a8009fb0 tx=0x7fe0a8011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:08.818 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 shutdown_connections 2023-12-14T18:19:08.818 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 --2- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7fe098038670 0x7fe09803ab20 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:08.818 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 --2- 172.21.15.93:0/3254623753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe0ac106430 0x7fe0ac199930 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:08.818 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 >> 172.21.15.93:0/3254623753 conn(0x7fe0ac0ffe30 msgr2=0x7fe0ac102160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:08.818 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 shutdown_connections 2023-12-14T18:19:08.818 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:08.447+0000 7fe0b339e700 1 -- 172.21.15.93:0/3254623753 wait complete. 2023-12-14T18:19:09.011 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:08 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:08.809+0000 7f101d043200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:19:09.011 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:08 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:08.919+0000 7f101d043200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:19:09.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:09.010+0000 7f101d043200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:19:09.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:09.097+0000 7f101d043200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:19:09.738 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:19:09.738 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 5, 2023-12-14T18:19:09.738 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-14T18:19:09.738 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.150+0000 7f93830a8700 1 Processor -- start 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.150+0000 7f93830a8700 1 -- start start 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.150+0000 7f93830a8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c105ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.150+0000 7f93830a8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c106410 con 0x7f937c105ac0 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.151+0000 7f9380e44700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c105ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.151+0000 7f9380e44700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c105ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33978/0 (socket says 172.21.15.93:33978) 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.151+0000 7f9380e44700 1 -- 172.21.15.93:0/2215799457 learned_addr learned my addr 172.21.15.93:0/2215799457 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.151+0000 7f9380e44700 1 -- 172.21.15.93:0/2215799457 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f937c106550 con 0x7f937c105ac0 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.151+0000 7f9380e44700 1 --2- 172.21.15.93:0/2215799457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c105ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f93700098a0 tx=0x7f9370009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a58ade6dea0cca2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.151+0000 7f937b7fe700 1 -- 172.21.15.93:0/2215799457 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9370004160 con 0x7f937c105ac0 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.151+0000 7f937b7fe700 1 -- 172.21.15.93:0/2215799457 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f93700042c0 con 0x7f937c105ac0 2023-12-14T18:19:09.739 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.151+0000 7f937b7fe700 1 -- 172.21.15.93:0/2215799457 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9370011a00 con 0x7f937c105ac0 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.152+0000 7f93830a8700 1 -- 172.21.15.93:0/2215799457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 msgr2=0x7f937c105ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.152+0000 7f93830a8700 1 --2- 172.21.15.93:0/2215799457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c105ed0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f93700098a0 tx=0x7f9370009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.152+0000 7f93830a8700 1 -- 172.21.15.93:0/2215799457 shutdown_connections 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.152+0000 7f93830a8700 1 --2- 172.21.15.93:0/2215799457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c105ed0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.152+0000 7f93830a8700 1 -- 172.21.15.93:0/2215799457 >> 172.21.15.93:0/2215799457 conn(0x7f937c101170 msgr2=0x7f937c103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.152+0000 7f93830a8700 1 -- 172.21.15.93:0/2215799457 shutdown_connections 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.152+0000 7f93830a8700 1 -- 172.21.15.93:0/2215799457 wait complete. 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.153+0000 7f93830a8700 1 Processor -- start 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.153+0000 7f93830a8700 1 -- start start 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.153+0000 7f93830a8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c199820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f93830a8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f937c199d60 con 0x7f937c105ac0 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f9380e44700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c199820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f9380e44700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c199820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33984/0 (socket says 172.21.15.93:33984) 2023-12-14T18:19:09.740 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f9380e44700 1 -- 172.21.15.93:0/1084124979 learned_addr learned my addr 172.21.15.93:0/1084124979 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f9380e44700 1 -- 172.21.15.93:0/1084124979 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9370009550 con 0x7f937c105ac0 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f9380e44700 1 --2- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c199820 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9370006af0 tx=0x7f9370011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f9379ffb700 1 -- 172.21.15.93:0/1084124979 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9370003a40 con 0x7f937c105ac0 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f9379ffb700 1 -- 172.21.15.93:0/1084124979 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f9370003ba0 con 0x7f937c105ac0 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f937c107b10 con 0x7f937c105ac0 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f9379ffb700 1 -- 172.21.15.93:0/1084124979 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f937001a6e0 con 0x7f937c105ac0 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.154+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f937c107980 con 0x7f937c105ac0 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.155+0000 7f9379ffb700 1 -- 172.21.15.93:0/1084124979 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f937001a950 con 0x7f937c105ac0 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.156+0000 7f9379ffb700 1 --2- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f936c038710 0x7f936c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.156+0000 7f9379ffb700 1 -- 172.21.15.93:0/1084124979 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f937004c710 con 0x7f937c105ac0 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.156+0000 7f937bfff700 1 -- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f936c038710 msgr2=0x7f936c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/1555415520 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.156+0000 7f937bfff700 1 --2- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f936c038710 0x7f936c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:19:09.741 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.156+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f937c109f90 con 0x7f937c105ac0 2023-12-14T18:19:09.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.161+0000 7f9379ffb700 1 -- 172.21.15.93:0/1084124979 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f937001ac00 con 0x7f937c105ac0 2023-12-14T18:19:09.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.357+0000 7f937bfff700 1 -- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f936c038710 msgr2=0x7f936c03abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/1555415520 2023-12-14T18:19:09.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.357+0000 7f937bfff700 1 --2- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f936c038710 0x7f936c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:19:09.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.427+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f937c062120 con 0x7f937c105ac0 2023-12-14T18:19:09.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.427+0000 7f9379ffb700 1 -- 172.21.15.93:0/1084124979 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) v1 ==== 56+0+88 (secure 0 0 0) 0x7f9370018350 con 0x7f937c105ac0 2023-12-14T18:19:09.742 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f936c038710 msgr2=0x7f936c03abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:19:09.742 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:09 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3254623753' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2023-12-14T18:19:09.742 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:09 smithi093 ceph-mon[42070]: mgrmap e5: x(active, since 9s) 2023-12-14T18:19:09.742 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:09 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1084124979' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-14T18:19:09.743 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 --2- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f936c038710 0x7f936c03abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:09.743 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 msgr2=0x7f937c199820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:09.743 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 --2- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c199820 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f9370006af0 tx=0x7f9370011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:09.743 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 shutdown_connections 2023-12-14T18:19:09.744 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 --2- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7f936c038710 0x7f936c03abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:09.744 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 --2- 172.21.15.93:0/1084124979 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f937c105ac0 0x7f937c199820 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:09.744 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 >> 172.21.15.93:0/1084124979 conn(0x7f937c101170 msgr2=0x7f937c101f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:09.744 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 shutdown_connections 2023-12-14T18:19:09.744 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:09.430+0000 7f93830a8700 1 -- 172.21.15.93:0/1084124979 wait complete. 2023-12-14T18:19:09.744 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for the mgr to restart... 2023-12-14T18:19:09.744 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr epoch 5... 2023-12-14T18:19:09.992 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:09.735+0000 7f101d043200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:19:10.282 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:09.991+0000 7f101d043200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:19:10.282 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:10.080+0000 7f101d043200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:19:10.282 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:10.193+0000 7f101d043200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:19:10.283 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:10.282+0000 7f101d043200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:19:10.557 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:10.473+0000 7f101d043200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:19:10.814 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:10.557+0000 7f101d043200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:19:11.186 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:10.923+0000 7f101d043200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:19:11.480 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:11 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:11.186+0000 7f101d043200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:19:11.814 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:11 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:11.479+0000 7f101d043200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:19:11.814 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:11 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:11.573+0000 7f101d043200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:19:12.814 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:12.334+0000 7f101d043200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:19:12.815 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:12.430+0000 7f101d043200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:19:12.815 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:12.576+0000 7f101d043200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:19:13.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:13 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:13.195+0000 7f101d043200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:19:13.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:13 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:13.374+0000 7f101d043200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:19:14.064 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:13 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:13.564+0000 7f101d043200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:19:14.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:14 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:14.271+0000 7f101d043200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:19:15.042 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:14 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:14.762+0000 7f101d043200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:19:15.042 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:14 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:14.850+0000 7f101d043200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:19:15.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:15.041+0000 7f101d043200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: Active manager daemon x restarted 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: Activating manager daemon x 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: osdmap e2: 0 total, 0 up, 0 in 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: mgrmap e6: x(active, starting, since 0.00556305s) 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: Manager daemon x is now available 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:15 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:16.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:16 smithi093 ceph-mon[42070]: Found migration_current of "None". Setting to last migration. 2023-12-14T18:19:16.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:16 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:16.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:16 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:16.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:16 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:19:16.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:16 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:19:16.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:16 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:16.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:16 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:16.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:16 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:16.385 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:16 smithi093 ceph-mon[42070]: mgrmap e7: x(active, since 1.00548s) 2023-12-14T18:19:16.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:19:16.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2023-12-14T18:19:16.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-14T18:19:16.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:19:16.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.099+0000 7ff2619b3700 1 Processor -- start 2023-12-14T18:19:16.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.099+0000 7ff2619b3700 1 -- start start 2023-12-14T18:19:16.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.099+0000 7ff2619b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c108030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:16.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.099+0000 7ff2619b3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c108600 con 0x7ff25c107c20 2023-12-14T18:19:16.423 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.099+0000 7ff25affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c108030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.099+0000 7ff25affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c108030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33992/0 (socket says 172.21.15.93:33992) 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.099+0000 7ff25affd700 1 -- 172.21.15.93:0/4001325989 learned_addr learned my addr 172.21.15.93:0/4001325989 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.100+0000 7ff25affd700 1 -- 172.21.15.93:0/4001325989 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c108e20 con 0x7ff25c107c20 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.100+0000 7ff25affd700 1 --2- 172.21.15.93:0/4001325989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c108030 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0098a0 tx=0x7ff24c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1d2a5a771e531 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.100+0000 7ff259ffb700 1 -- 172.21.15.93:0/4001325989 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c004160 con 0x7ff25c107c20 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.100+0000 7ff259ffb700 1 -- 172.21.15.93:0/4001325989 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff24c0042c0 con 0x7ff25c107c20 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.100+0000 7ff259ffb700 1 -- 172.21.15.93:0/4001325989 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c011a00 con 0x7ff25c107c20 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.100+0000 7ff2619b3700 1 -- 172.21.15.93:0/4001325989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 msgr2=0x7ff25c108030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.100+0000 7ff2619b3700 1 --2- 172.21.15.93:0/4001325989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c108030 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff24c0098a0 tx=0x7ff24c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.101+0000 7ff2619b3700 1 -- 172.21.15.93:0/4001325989 shutdown_connections 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.101+0000 7ff2619b3700 1 --2- 172.21.15.93:0/4001325989 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c108030 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.101+0000 7ff2619b3700 1 -- 172.21.15.93:0/4001325989 >> 172.21.15.93:0/4001325989 conn(0x7ff25c07b000 msgr2=0x7ff25c07b420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.101+0000 7ff2619b3700 1 -- 172.21.15.93:0/4001325989 shutdown_connections 2023-12-14T18:19:16.424 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.101+0000 7ff2619b3700 1 -- 172.21.15.93:0/4001325989 wait complete. 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.101+0000 7ff2619b3700 1 Processor -- start 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.101+0000 7ff2619b3700 1 -- start start 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff2619b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c19bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff2619b3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c19c070 con 0x7ff25c107c20 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff25affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c19bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff25affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c19bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33994/0 (socket says 172.21.15.93:33994) 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff25affd700 1 -- 172.21.15.93:0/1614561875 learned_addr learned my addr 172.21.15.93:0/1614561875 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff25affd700 1 -- 172.21.15.93:0/1614561875 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24c009550 con 0x7ff25c107c20 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff25affd700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c19bb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006af0 tx=0x7ff24c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c003a40 con 0x7ff25c107c20 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff25c10c050 con 0x7ff25c107c20 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7ff24c003ba0 con 0x7ff25c107c20 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff24c01a700 con 0x7ff25c107c20 2023-12-14T18:19:16.425 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.102+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff25c10c1e0 con 0x7ff25c107c20 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.103+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 5) v1 ==== 45483+0+0 (secure 0 0 0) 0x7ff24c01a980 con 0x7ff25c107c20 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.103+0000 7ff2609b1700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 0x7ff24403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.103+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 --> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff25c10c370 con 0x7ff244038760 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.103+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7ff24c04d6a0 con 0x7ff25c107c20 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.103+0000 7ff25a7fc700 1 -- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 msgr2=0x7ff24403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/1555415520 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.103+0000 7ff25a7fc700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 0x7ff24403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.304+0000 7ff25a7fc700 1 -- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 msgr2=0x7ff24403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/1555415520 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.304+0000 7ff25a7fc700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 0x7ff24403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.705+0000 7ff25a7fc700 1 -- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 msgr2=0x7ff24403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/1555415520 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:10.705+0000 7ff25a7fc700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 0x7ff24403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:11.506+0000 7ff25a7fc700 1 -- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 msgr2=0x7ff24403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/1555415520 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:11.506+0000 7ff25a7fc700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 0x7ff24403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:13.107+0000 7ff25a7fc700 1 -- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 msgr2=0x7ff24403ac10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/1555415520 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:13.107+0000 7ff25a7fc700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 0x7ff24403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-14T18:19:16.426 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:15.058+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mgrmap(e 6) v1 ==== 45250+0+0 (secure 0 0 0) 0x7ff24c02d960 con 0x7ff25c107c20 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:15.058+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 msgr2=0x7ff24403ac10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:15.058+0000 7ff2609b1700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 0x7ff24403ac10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.057+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7ff24c024ad0 con 0x7ff25c107c20 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.057+0000 7ff2609b1700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7ff24403be80 0x7ff24403e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.057+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff25c10c370 con 0x7ff24403be80 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.058+0000 7ff25a7fc700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7ff24403be80 0x7ff24403e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.058+0000 7ff25a7fc700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7ff24403be80 0x7ff24403e260 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff250003a10 tx=0x7ff2500092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.059+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 <== mgr.14120 v2:172.21.15.93:6800/2647964405 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7ff25c10c370 con 0x7ff24403be80 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7ff25c19ce70 con 0x7ff24403be80 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2609b1700 1 -- 172.21.15.93:0/1614561875 <== mgr.14120 v2:172.21.15.93:6800/2647964405 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff25c19ce70 con 0x7ff24403be80 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7ff24403be80 msgr2=0x7ff24403e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7ff24403be80 0x7ff24403e260 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff250003a10 tx=0x7ff2500092b0 comp rx=0 tx=0).stop 2023-12-14T18:19:16.427 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 msgr2=0x7ff25c19bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c19bb30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff24c006af0 tx=0x7ff24c011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 shutdown_connections 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7ff24403be80 0x7ff24403e260 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff25c107c20 0x7ff25c19bb30 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 --2- 172.21.15.93:0/1614561875 >> [v2:172.21.15.93:6800/1555415520,v1:172.21.15.93:6801/1555415520] conn(0x7ff244038760 0x7ff24403ac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 >> 172.21.15.93:0/1614561875 conn(0x7ff25c07b000 msgr2=0x7ff25c105840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.066+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 shutdown_connections 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.067+0000 7ff2619b3700 1 -- 172.21.15.93:0/1614561875 wait complete. 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:mgr epoch 5 is available 2023-12-14T18:19:16.428 INFO:teuthology.orchestra.run.smithi093.stdout:Setting orchestrator backend to cephadm... 2023-12-14T18:19:17.276 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:17 smithi093 ceph-mon[42070]: [14/Dec/2023:18:19:15] ENGINE Bus STARTING 2023-12-14T18:19:17.277 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:17 smithi093 ceph-mon[42070]: [14/Dec/2023:18:19:15] ENGINE Serving on http://172.21.15.93:8765 2023-12-14T18:19:17.277 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:17 smithi093 ceph-mon[42070]: [14/Dec/2023:18:19:15] ENGINE Serving on https://172.21.15.93:7150 2023-12-14T18:19:17.277 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:17 smithi093 ceph-mon[42070]: [14/Dec/2023:18:19:15] ENGINE Bus STARTED 2023-12-14T18:19:17.277 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:17 smithi093 ceph-mon[42070]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-14T18:19:17.277 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:17 smithi093 ceph-mon[42070]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-14T18:19:17.277 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:17 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:17.277 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:17 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:17.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.748+0000 7f201829b700 1 Processor -- start 2023-12-14T18:19:17.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.748+0000 7f201829b700 1 -- start start 2023-12-14T18:19:17.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.748+0000 7f201829b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f2010104710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:17.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.748+0000 7f201829b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2010079de0 con 0x7f2010104300 2023-12-14T18:19:17.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.748+0000 7f2016037700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f2010104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:17.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.748+0000 7f2016037700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f2010104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57026/0 (socket says 172.21.15.93:57026) 2023-12-14T18:19:17.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.748+0000 7f2016037700 1 -- 172.21.15.93:0/1300074435 learned_addr learned my addr 172.21.15.93:0/1300074435 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:17.346 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.749+0000 7f2016037700 1 -- 172.21.15.93:0/1300074435 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2010104c50 con 0x7f2010104300 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.749+0000 7f2016037700 1 --2- 172.21.15.93:0/1300074435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f2010104710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f200c0098a0 tx=0x7f200c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5bbbdfd9a806b61 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.749+0000 7f2015035700 1 -- 172.21.15.93:0/1300074435 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f200c004160 con 0x7f2010104300 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.749+0000 7f2015035700 1 -- 172.21.15.93:0/1300074435 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f200c0042c0 con 0x7f2010104300 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.749+0000 7f2015035700 1 -- 172.21.15.93:0/1300074435 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f200c011a00 con 0x7f2010104300 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.749+0000 7f201829b700 1 -- 172.21.15.93:0/1300074435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 msgr2=0x7f2010104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.749+0000 7f201829b700 1 --2- 172.21.15.93:0/1300074435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f2010104710 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f200c0098a0 tx=0x7f200c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f201829b700 1 -- 172.21.15.93:0/1300074435 shutdown_connections 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f201829b700 1 --2- 172.21.15.93:0/1300074435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f2010104710 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f201829b700 1 -- 172.21.15.93:0/1300074435 >> 172.21.15.93:0/1300074435 conn(0x7f20100ffed0 msgr2=0x7f2010102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f201829b700 1 -- 172.21.15.93:0/1300074435 shutdown_connections 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f201829b700 1 -- 172.21.15.93:0/1300074435 wait complete. 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f201829b700 1 Processor -- start 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f201829b700 1 -- start start 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f201829b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f201019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f201829b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f201019beb0 con 0x7f2010104300 2023-12-14T18:19:17.347 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f2016037700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f201019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f2016037700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f201019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57028/0 (socket says 172.21.15.93:57028) 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.750+0000 7f2016037700 1 -- 172.21.15.93:0/2247263323 learned_addr learned my addr 172.21.15.93:0/2247263323 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.751+0000 7f2016037700 1 -- 172.21.15.93:0/2247263323 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f200c009550 con 0x7f2010104300 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.751+0000 7f2016037700 1 --2- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f201019b970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f200c009fb0 tx=0x7f200c011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.751+0000 7f20077fe700 1 -- 172.21.15.93:0/2247263323 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f200c003930 con 0x7f2010104300 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.751+0000 7f20077fe700 1 -- 172.21.15.93:0/2247263323 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f200c003a90 con 0x7f2010104300 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.751+0000 7f20077fe700 1 -- 172.21.15.93:0/2247263323 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f200c019610 con 0x7f2010104300 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.751+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2010105c50 con 0x7f2010104300 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.751+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2010105ac0 con 0x7f2010104300 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.752+0000 7f20077fe700 1 -- 172.21.15.93:0/2247263323 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7f200c01a430 con 0x7f2010104300 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.752+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2010195240 con 0x7f2010104300 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.752+0000 7f20077fe700 1 --2- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f1ffc0385f0 0x7f1ffc03aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.752+0000 7f20077fe700 1 -- 172.21.15.93:0/2247263323 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f200c04c620 con 0x7f2010104300 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.752+0000 7f2015836700 1 --2- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f1ffc0385f0 0x7f1ffc03aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:17.348 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.752+0000 7f2015836700 1 --2- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f1ffc0385f0 0x7f1ffc03aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2000006fd0 tx=0x7f2000006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.757+0000 7f20077fe700 1 -- 172.21.15.93:0/2247263323 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f200c018350 con 0x7f2010104300 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.957+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f2010060f30 con 0x7f1ffc0385f0 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.963+0000 7f20077fe700 1 -- 172.21.15.93:0/2247263323 <== mgr.14120 v2:172.21.15.93:6800/2647964405 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f2010060f30 con 0x7f1ffc0385f0 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f1ffc0385f0 msgr2=0x7f1ffc03aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 --2- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f1ffc0385f0 0x7f1ffc03aaa0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2000006fd0 tx=0x7f2000006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 msgr2=0x7f201019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 --2- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f201019b970 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f200c009fb0 tx=0x7f200c011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 shutdown_connections 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 --2- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f1ffc0385f0 0x7f1ffc03aaa0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 --2- 172.21.15.93:0/2247263323 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2010104300 0x7f201019b970 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 >> 172.21.15.93:0/2247263323 conn(0x7f20100ffed0 msgr2=0x7f201018fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 shutdown_connections 2023-12-14T18:19:17.349 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:16.965+0000 7f201829b700 1 -- 172.21.15.93:0/2247263323 wait complete. 2023-12-14T18:19:18.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:18 smithi093 ceph-mon[42070]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:18.409 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:18 smithi093 ceph-mon[42070]: mgrmap e8: x(active, since 2s) 2023-12-14T18:19:18.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout value unchanged 2023-12-14T18:19:18.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.889+0000 7fe78cd8b700 1 Processor -- start 2023-12-14T18:19:18.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.889+0000 7fe78cd8b700 1 -- start start 2023-12-14T18:19:18.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.890+0000 7fe78cd8b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe7881088b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:18.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.890+0000 7fe78cd8b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788079e20 con 0x7fe7881064d0 2023-12-14T18:19:18.495 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.890+0000 7fe78659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe7881088b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.890+0000 7fe78659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe7881088b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57036/0 (socket says 172.21.15.93:57036) 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.890+0000 7fe78659c700 1 -- 172.21.15.93:0/2267501480 learned_addr learned my addr 172.21.15.93:0/2267501480 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.890+0000 7fe78659c700 1 -- 172.21.15.93:0/2267501480 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe788108df0 con 0x7fe7881064d0 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.890+0000 7fe78659c700 1 --2- 172.21.15.93:0/2267501480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe7881088b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7780098a0 tx=0x7fe778009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cb09c684b171f33 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.890+0000 7fe78559a700 1 -- 172.21.15.93:0/2267501480 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe778004160 con 0x7fe7881064d0 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.891+0000 7fe78559a700 1 -- 172.21.15.93:0/2267501480 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe7780042c0 con 0x7fe7881064d0 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.891+0000 7fe78559a700 1 -- 172.21.15.93:0/2267501480 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe778011a00 con 0x7fe7881064d0 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.891+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2267501480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 msgr2=0x7fe7881088b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.891+0000 7fe78cd8b700 1 --2- 172.21.15.93:0/2267501480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe7881088b0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fe7780098a0 tx=0x7fe778009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.891+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2267501480 shutdown_connections 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.891+0000 7fe78cd8b700 1 --2- 172.21.15.93:0/2267501480 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe7881088b0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.891+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2267501480 >> 172.21.15.93:0/2267501480 conn(0x7fe7880ffed0 msgr2=0x7fe788102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.891+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2267501480 shutdown_connections 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.891+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2267501480 wait complete. 2023-12-14T18:19:18.496 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.892+0000 7fe78cd8b700 1 Processor -- start 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.892+0000 7fe78cd8b700 1 -- start start 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.892+0000 7fe78cd8b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe788105e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.892+0000 7fe78cd8b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe788106370 con 0x7fe7881064d0 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.892+0000 7fe78659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe788105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.892+0000 7fe78659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe788105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57044/0 (socket says 172.21.15.93:57044) 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.892+0000 7fe78659c700 1 -- 172.21.15.93:0/2375758435 learned_addr learned my addr 172.21.15.93:0/2375758435 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.892+0000 7fe78659c700 1 -- 172.21.15.93:0/2375758435 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe778009550 con 0x7fe7881064d0 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.893+0000 7fe78659c700 1 --2- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe788105e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe778006af0 tx=0x7fe778011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.893+0000 7fe7777fe700 1 -- 172.21.15.93:0/2375758435 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe778003a40 con 0x7fe7881064d0 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.893+0000 7fe7777fe700 1 -- 172.21.15.93:0/2375758435 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7fe778003ba0 con 0x7fe7881064d0 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.893+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe788109d20 con 0x7fe7881064d0 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.893+0000 7fe7777fe700 1 -- 172.21.15.93:0/2375758435 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe77801a6e0 con 0x7fe7881064d0 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.893+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe788109b90 con 0x7fe7881064d0 2023-12-14T18:19:18.497 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.894+0000 7fe7777fe700 1 -- 172.21.15.93:0/2375758435 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 7) v1 ==== 45377+0+0 (secure 0 0 0) 0x7fe77801a950 con 0x7fe7881064d0 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.894+0000 7fe7777fe700 1 --2- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fe7700385e0 0x7fe77003aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.894+0000 7fe7777fe700 1 -- 172.21.15.93:0/2375758435 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fe77804c740 con 0x7fe7881064d0 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.894+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe788062120 con 0x7fe7881064d0 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.894+0000 7fe785d9b700 1 --2- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fe7700385e0 0x7fe77003aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.894+0000 7fe785d9b700 1 --2- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fe7700385e0 0x7fe77003aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.898+0000 7fe7777fe700 1 -- 172.21.15.93:0/2375758435 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe7780189a0 con 0x7fe7881064d0 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:17.962+0000 7fe7777fe700 1 -- 172.21.15.93:0/2375758435 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7fe77802d4e0 con 0x7fe7881064d0 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.091+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7881052e0 con 0x7fe7700385e0 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.092+0000 7fe7777fe700 1 -- 172.21.15.93:0/2375758435 <== mgr.14120 v2:172.21.15.93:6800/2647964405 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7fe7881052e0 con 0x7fe7700385e0 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fe7700385e0 msgr2=0x7fe77003aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 --2- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fe7700385e0 0x7fe77003aa90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fe77c006fd0 tx=0x7fe77c006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 msgr2=0x7fe788105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 --2- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe788105e30 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fe778006af0 tx=0x7fe778011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 shutdown_connections 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 --2- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fe7700385e0 0x7fe77003aa90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:18.498 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 --2- 172.21.15.93:0/2375758435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe7881064d0 0x7fe788105e30 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:18.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 >> 172.21.15.93:0/2375758435 conn(0x7fe7880ffed0 msgr2=0x7fe788101530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:18.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 shutdown_connections 2023-12-14T18:19:18.499 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.094+0000 7fe78cd8b700 1 -- 172.21.15.93:0/2375758435 wait complete. 2023-12-14T18:19:18.499 INFO:teuthology.orchestra.run.smithi093.stdout:Generating ssh key... 2023-12-14T18:19:19.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-mon[42070]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:19.759 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: Generating public/private rsa key pair. 2023-12-14T18:19:19.759 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: Your identification has been saved in /tmp/tmp66kjp82w/key. 2023-12-14T18:19:19.759 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: Your public key has been saved in /tmp/tmp66kjp82w/key.pub. 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: The key fingerprint is: 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: SHA256:TQ4zfhge6ATPZ/LSbMEhFRgHKNa908TM2zEJxiwGEgk ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: The key's randomart image is: 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: +---[RSA 3072]----+ 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: |E.o+o++@Bo . | 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: | +.o+=*B.+ | 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: | . . .*=@o.o | 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: | ooXo%. | 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: | o.S + | 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: | o . | 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: | | 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: | | 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: | | 2023-12-14T18:19:19.760 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: +----[SHA256]-----+ 2023-12-14T18:19:19.854 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.878+0000 7f56baa97700 1 Processor -- start 2023-12-14T18:19:19.854 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.878+0000 7f56baa97700 1 -- start start 2023-12-14T18:19:19.854 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.878+0000 7f56baa97700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b4104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:19.854 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.878+0000 7f56baa97700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4105300 con 0x7f56b4104920 2023-12-14T18:19:19.854 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.878+0000 7f56b8833700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b4104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:19.854 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.878+0000 7f56b8833700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b4104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57056/0 (socket says 172.21.15.93:57056) 2023-12-14T18:19:19.854 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.878+0000 7f56b8833700 1 -- 172.21.15.93:0/3664337569 learned_addr learned my addr 172.21.15.93:0/3664337569 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:19.854 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.878+0000 7f56b8833700 1 -- 172.21.15.93:0/3664337569 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b4105b20 con 0x7f56b4104920 2023-12-14T18:19:19.854 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.878+0000 7f56b8833700 1 --2- 172.21.15.93:0/3664337569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b4104d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f56a80098a0 tx=0x7f56a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6d02c815ef6f13d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56b37fe700 1 -- 172.21.15.93:0/3664337569 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56a8004160 con 0x7f56b4104920 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56b37fe700 1 -- 172.21.15.93:0/3664337569 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f56a80042c0 con 0x7f56b4104920 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56b37fe700 1 -- 172.21.15.93:0/3664337569 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56a8011a00 con 0x7f56b4104920 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56baa97700 1 -- 172.21.15.93:0/3664337569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 msgr2=0x7f56b4104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56baa97700 1 --2- 172.21.15.93:0/3664337569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b4104d30 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f56a80098a0 tx=0x7f56a8009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56baa97700 1 -- 172.21.15.93:0/3664337569 shutdown_connections 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56baa97700 1 --2- 172.21.15.93:0/3664337569 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b4104d30 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56baa97700 1 -- 172.21.15.93:0/3664337569 >> 172.21.15.93:0/3664337569 conn(0x7f56b40ffed0 msgr2=0x7f56b4102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56baa97700 1 -- 172.21.15.93:0/3664337569 shutdown_connections 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.879+0000 7f56baa97700 1 -- 172.21.15.93:0/3664337569 wait complete. 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.880+0000 7f56baa97700 1 Processor -- start 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.880+0000 7f56baa97700 1 -- start start 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.880+0000 7f56baa97700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b41975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.880+0000 7f56baa97700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b4197ae0 con 0x7f56b4104920 2023-12-14T18:19:19.855 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.880+0000 7f56b8833700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b41975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.880+0000 7f56b8833700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b41975a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57062/0 (socket says 172.21.15.93:57062) 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.880+0000 7f56b8833700 1 -- 172.21.15.93:0/4079266695 learned_addr learned my addr 172.21.15.93:0/4079266695 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.880+0000 7f56b8833700 1 -- 172.21.15.93:0/4079266695 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a8009550 con 0x7f56b4104920 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.880+0000 7f56b8833700 1 --2- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b41975a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009fb0 tx=0x7f56a8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.881+0000 7f56b1ffb700 1 -- 172.21.15.93:0/4079266695 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56a8003930 con 0x7f56b4104920 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.881+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b4078f80 con 0x7f56b4104920 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.881+0000 7f56b1ffb700 1 -- 172.21.15.93:0/4079266695 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f56a8003a90 con 0x7f56b4104920 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.881+0000 7f56b1ffb700 1 -- 172.21.15.93:0/4079266695 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f56a80195e0 con 0x7f56b4104920 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.881+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b41068c0 con 0x7f56b4104920 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.882+0000 7f56b1ffb700 1 -- 172.21.15.93:0/4079266695 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f56a80198a0 con 0x7f56b4104920 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.882+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b41911b0 con 0x7f56b4104920 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.882+0000 7f56b1ffb700 1 --2- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f56a403a8c0 0x7f56a403cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.882+0000 7f56b1ffb700 1 -- 172.21.15.93:0/4079266695 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f56a804ca80 con 0x7f56b4104920 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.882+0000 7f56b3fff700 1 --2- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f56a403a8c0 0x7f56a403cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:19.856 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.883+0000 7f56b3fff700 1 --2- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f56a403a8c0 0x7f56a403cd70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56a0006fd0 tx=0x7f56a0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:18.887+0000 7f56b1ffb700 1 -- 172.21.15.93:0/4079266695 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f56a802b340 con 0x7f56b4104920 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.077+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f56b4060f30 con 0x7f56a403a8c0 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.479+0000 7f56b1ffb700 1 -- 172.21.15.93:0/4079266695 <== mgr.14120 v2:172.21.15.93:6800/2647964405 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f56b4060f30 con 0x7f56a403a8c0 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f56a403a8c0 msgr2=0x7f56a403cd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 --2- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f56a403a8c0 0x7f56a403cd70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f56a0006fd0 tx=0x7f56a0006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 msgr2=0x7f56b41975a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 --2- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b41975a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f56a8009fb0 tx=0x7f56a8011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 shutdown_connections 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 --2- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f56a403a8c0 0x7f56a403cd70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 --2- 172.21.15.93:0/4079266695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b4104920 0x7f56b41975a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 >> 172.21.15.93:0/4079266695 conn(0x7f56b40ffed0 msgr2=0x7f56b418e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 shutdown_connections 2023-12-14T18:19:19.857 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:19.481+0000 7f56baa97700 1 -- 172.21.15.93:0/4079266695 wait complete. 2023-12-14T18:19:20.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:20 smithi093 ceph-mon[42070]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:20.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:20 smithi093 ceph-mon[42070]: Generating ssh key... 2023-12-14T18:19:20.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:20 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:20.651 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:20 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwpfVZRML/QmaCwEm4CeUHs1hwpffM4F7L7DbjZ3c1z3igmNNVbxQefHJGjjMdKb0RoEiOp/e6WuisxpqxqNgwEfGWfAMP0X3qT+V6yN8kddgyxfX/hmJbsRnRMUBQ07Mz148we93kWiqOrf4W2JQfr3sPbGjB+ibgil+0/3ZZSovq98lOwA4sMmDOuy3p5Y0Nl7UfUe0g4Pj7a051yxcHzdKRKpUo//yyXYnjpmkZvw1K0w6aED5aFniZh3NG4hfEvFJe78/mUrs9J+5sNZowOppBGRf6NDVaLbt1a6mlx5ZxOVU+KoSkC04o2fhABIDYoT464g9oNVuickr0dT1p5MmO6odd3lFVIsxJmda5quECJlHf2HUUfmZ1hTm8mReq2NCo931X5M5PG8GK7GL1E4wbmVsU2IUnQRPo6RT1bo6fdeo6QyxwQh0G3n3a1+uX3qGvJtZNH5VE8g4ezAQzo4DgsIFs070F/w0YO37JYw0oN+V2E9DRYQlPbl8hgdk= ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.171+0000 7f03a67e6700 1 Processor -- start 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.171+0000 7f03a67e6700 1 -- start start 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.171+0000 7f03a67e6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0105e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.171+0000 7f03a67e6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0106400 con 0x7f03a0105a20 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.171+0000 7f039ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.171+0000 7f039ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57068/0 (socket says 172.21.15.93:57068) 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.171+0000 7f039ffff700 1 -- 172.21.15.93:0/2111613170 learned_addr learned my addr 172.21.15.93:0/2111613170 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.171+0000 7f039ffff700 1 -- 172.21.15.93:0/2111613170 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03a0106c20 con 0x7f03a0105a20 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.171+0000 7f039ffff700 1 --2- 172.21.15.93:0/2111613170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0105e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03900098a0 tx=0x7f0390009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74b43ea5f82acdf0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f039effd700 1 -- 172.21.15.93:0/2111613170 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0390004160 con 0x7f03a0105a20 2023-12-14T18:19:20.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f039effd700 1 -- 172.21.15.93:0/2111613170 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f03900042c0 con 0x7f03a0105a20 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f039effd700 1 -- 172.21.15.93:0/2111613170 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0390011a00 con 0x7f03a0105a20 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f03a67e6700 1 -- 172.21.15.93:0/2111613170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 msgr2=0x7f03a0105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f03a67e6700 1 --2- 172.21.15.93:0/2111613170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0105e30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f03900098a0 tx=0x7f0390009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f03a67e6700 1 -- 172.21.15.93:0/2111613170 shutdown_connections 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f03a67e6700 1 --2- 172.21.15.93:0/2111613170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0105e30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f03a67e6700 1 -- 172.21.15.93:0/2111613170 >> 172.21.15.93:0/2111613170 conn(0x7f03a0100fd0 msgr2=0x7f03a0103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f03a67e6700 1 -- 172.21.15.93:0/2111613170 shutdown_connections 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.172+0000 7f03a67e6700 1 -- 172.21.15.93:0/2111613170 wait complete. 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.173+0000 7f03a67e6700 1 Processor -- start 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.173+0000 7f03a67e6700 1 -- start start 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.173+0000 7f03a67e6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f03a67e6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03a0199e70 con 0x7f03a0105a20 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f039ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:20.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f039ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0199930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57076/0 (socket says 172.21.15.93:57076) 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f039ffff700 1 -- 172.21.15.93:0/767654394 learned_addr learned my addr 172.21.15.93:0/767654394 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f039ffff700 1 -- 172.21.15.93:0/767654394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0390009550 con 0x7f03a0105a20 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f039ffff700 1 --2- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0199930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0390006af0 tx=0x7f0390011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f039d7fa700 1 -- 172.21.15.93:0/767654394 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0390003a40 con 0x7f03a0105a20 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f039d7fa700 1 -- 172.21.15.93:0/767654394 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f0390003ba0 con 0x7f03a0105a20 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03a0109e50 con 0x7f03a0105a20 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f039d7fa700 1 -- 172.21.15.93:0/767654394 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f039001a700 con 0x7f03a0105a20 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.174+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03a0109fe0 con 0x7f03a0105a20 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.175+0000 7f039d7fa700 1 -- 172.21.15.93:0/767654394 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f039001a980 con 0x7f03a0105a20 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.175+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03a0107a90 con 0x7f03a0105a20 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.175+0000 7f039d7fa700 1 --2- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0388038710 0x7f038803abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.175+0000 7f039d7fa700 1 -- 172.21.15.93:0/767654394 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f039004c8c0 con 0x7f03a0105a20 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.175+0000 7f039f7fe700 1 --2- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0388038710 0x7f038803abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.176+0000 7f039f7fe700 1 --2- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0388038710 0x7f038803abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0394006fd0 tx=0x7f0394006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:20.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.180+0000 7f039d7fa700 1 -- 172.21.15.93:0/767654394 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f03a0107a90 con 0x7f03a0105a20 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.367+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f03a0107a90 con 0x7f0388038710 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.368+0000 7f039d7fa700 1 -- 172.21.15.93:0/767654394 <== mgr.14120 v2:172.21.15.93:6800/2647964405 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f03a0107a90 con 0x7f0388038710 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0388038710 msgr2=0x7f038803abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 --2- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0388038710 0x7f038803abc0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0394006fd0 tx=0x7f0394006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 msgr2=0x7f03a0199930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 --2- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0199930 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0390006af0 tx=0x7f0390011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 shutdown_connections 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 --2- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0388038710 0x7f038803abc0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 --2- 172.21.15.93:0/767654394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f03a0105a20 0x7f03a0199930 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 >> 172.21.15.93:0/767654394 conn(0x7f03a0100fd0 msgr2=0x7f03a0102640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 shutdown_connections 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:20.370+0000 7f03a67e6700 1 -- 172.21.15.93:0/767654394 wait complete. 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:Adding key to root@localhost authorized_keys... 2023-12-14T18:19:20.715 INFO:teuthology.orchestra.run.smithi093.stdout:Adding host smithi093... 2023-12-14T18:19:21.521 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:21 smithi093 ceph-mon[42070]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:22.792 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:22 smithi093 ceph-mon[42070]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi093", "addr": "172.21.15.93", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:23.501 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:23 smithi093 ceph-mon[42070]: Deploying cephadm binary to smithi093 2023-12-14T18:19:23.501 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:23 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:23.501 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:23 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:23.535 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Added host 'smithi093' with addr '172.21.15.93' 2023-12-14T18:19:23.535 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.048+0000 7f49c47e3700 1 Processor -- start 2023-12-14T18:19:23.535 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.048+0000 7f49c47e3700 1 -- start start 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.048+0000 7f49c47e3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc104710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.048+0000 7f49c47e3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc079de0 con 0x7f49bc104300 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.049+0000 7f49c257f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.049+0000 7f49c257f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57084/0 (socket says 172.21.15.93:57084) 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.049+0000 7f49c257f700 1 -- 172.21.15.93:0/948742527 learned_addr learned my addr 172.21.15.93:0/948742527 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.049+0000 7f49c257f700 1 -- 172.21.15.93:0/948742527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc104c50 con 0x7f49bc104300 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.049+0000 7f49c257f700 1 --2- 172.21.15.93:0/948742527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc104710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49b80098a0 tx=0x7f49b8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b38589af3a2b471 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.049+0000 7f49c157d700 1 -- 172.21.15.93:0/948742527 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49b8004160 con 0x7f49bc104300 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.049+0000 7f49c157d700 1 -- 172.21.15.93:0/948742527 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49b80042c0 con 0x7f49bc104300 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.049+0000 7f49c157d700 1 -- 172.21.15.93:0/948742527 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49b8011a00 con 0x7f49bc104300 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 -- 172.21.15.93:0/948742527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 msgr2=0x7f49bc104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 --2- 172.21.15.93:0/948742527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc104710 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f49b80098a0 tx=0x7f49b8009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 -- 172.21.15.93:0/948742527 shutdown_connections 2023-12-14T18:19:23.536 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 --2- 172.21.15.93:0/948742527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc104710 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 -- 172.21.15.93:0/948742527 >> 172.21.15.93:0/948742527 conn(0x7f49bc0ffed0 msgr2=0x7f49bc102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 -- 172.21.15.93:0/948742527 shutdown_connections 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 -- 172.21.15.93:0/948742527 wait complete. 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 Processor -- start 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 -- start start 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc19b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.050+0000 7f49c47e3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49bc19bee0 con 0x7f49bc104300 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49c257f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc19b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49c257f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc19b9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:57100/0 (socket says 172.21.15.93:57100) 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49c257f700 1 -- 172.21.15.93:0/559061040 learned_addr learned my addr 172.21.15.93:0/559061040 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49c257f700 1 -- 172.21.15.93:0/559061040 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b8009550 con 0x7f49bc104300 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49c257f700 1 --2- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc19b9a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009fb0 tx=0x7f49b8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49b37fe700 1 -- 172.21.15.93:0/559061040 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49b8003930 con 0x7f49bc104300 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49b37fe700 1 -- 172.21.15.93:0/559061040 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7f49b8003a90 con 0x7f49bc104300 2023-12-14T18:19:23.537 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49b37fe700 1 -- 172.21.15.93:0/559061040 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f49b8019610 con 0x7f49bc104300 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49bc105c50 con 0x7f49bc104300 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.051+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49bc105ac0 con 0x7f49bc104300 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.052+0000 7f49b37fe700 1 -- 172.21.15.93:0/559061040 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f49b801a9d0 con 0x7f49bc104300 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.052+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49bc195250 con 0x7f49bc104300 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.052+0000 7f49b37fe700 1 --2- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f49a80386c0 0x7f49a803ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.052+0000 7f49b37fe700 1 -- 172.21.15.93:0/559061040 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f49b804c820 con 0x7f49bc104300 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.052+0000 7f49c1d7e700 1 --2- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f49a80386c0 0x7f49a803ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.053+0000 7f49c1d7e700 1 --2- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f49a80386c0 0x7f49a803ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f49ac006fd0 tx=0x7f49ac006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.057+0000 7f49b37fe700 1 -- 172.21.15.93:0/559061040 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f49b8003c00 con 0x7f49bc104300 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:21.249+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi093", "addr": "172.21.15.93", "target": ["mon-mgr", ""]}) v1 -- 0x7f49bc060f30 con 0x7f49a80386c0 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.200+0000 7f49b37fe700 1 -- 172.21.15.93:0/559061040 <== mgr.14120 v2:172.21.15.93:6800/2647964405 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f49bc060f30 con 0x7f49a80386c0 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f49a80386c0 msgr2=0x7f49a803ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 --2- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f49a80386c0 0x7f49a803ab70 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f49ac006fd0 tx=0x7f49ac006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:23.538 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 msgr2=0x7f49bc19b9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:23.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 --2- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc19b9a0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f49b8009fb0 tx=0x7f49b8011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:23.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 shutdown_connections 2023-12-14T18:19:23.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 --2- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f49a80386c0 0x7f49a803ab70 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:23.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 --2- 172.21.15.93:0/559061040 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49bc104300 0x7f49bc19b9a0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:23.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 >> 172.21.15.93:0/559061040 conn(0x7f49bc0ffed0 msgr2=0x7f49bc18fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:23.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 shutdown_connections 2023-12-14T18:19:23.539 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.203+0000 7f49c47e3700 1 -- 172.21.15.93:0/559061040 wait complete. 2023-12-14T18:19:23.539 INFO:teuthology.orchestra.run.smithi093.stdout:Deploying unmanaged mon service... 2023-12-14T18:19:24.560 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.893+0000 7efffb558700 1 Processor -- start 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.894+0000 7efffb558700 1 -- start start 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.894+0000 7efffb558700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff4106300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.894+0000 7efffb558700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff41068d0 con 0x7efff4105ef0 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.894+0000 7efff92f4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff4106300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.894+0000 7efff92f4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff4106300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56568/0 (socket says 172.21.15.93:56568) 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.894+0000 7efff92f4700 1 -- 172.21.15.93:0/3093396270 learned_addr learned my addr 172.21.15.93:0/3093396270 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.894+0000 7efff92f4700 1 -- 172.21.15.93:0/3093396270 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efff4104b50 con 0x7efff4105ef0 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7efff92f4700 1 --2- 172.21.15.93:0/3093396270 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff4106300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7effe40098a0 tx=0x7effe4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fe55762d403b075 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7effebfff700 1 -- 172.21.15.93:0/3093396270 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe4004160 con 0x7efff4105ef0 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7effebfff700 1 -- 172.21.15.93:0/3093396270 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7effe40042c0 con 0x7efff4105ef0 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7effebfff700 1 -- 172.21.15.93:0/3093396270 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe4011a00 con 0x7efff4105ef0 2023-12-14T18:19:24.561 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7efffb558700 1 -- 172.21.15.93:0/3093396270 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 msgr2=0x7efff4106300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7efffb558700 1 --2- 172.21.15.93:0/3093396270 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff4106300 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7effe40098a0 tx=0x7effe4009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7efffb558700 1 -- 172.21.15.93:0/3093396270 shutdown_connections 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7efffb558700 1 --2- 172.21.15.93:0/3093396270 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff4106300 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7efffb558700 1 -- 172.21.15.93:0/3093396270 >> 172.21.15.93:0/3093396270 conn(0x7efff40ff650 msgr2=0x7efff4101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7efffb558700 1 -- 172.21.15.93:0/3093396270 shutdown_connections 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.895+0000 7efffb558700 1 -- 172.21.15.93:0/3093396270 wait complete. 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.896+0000 7efffb558700 1 Processor -- start 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.896+0000 7efffb558700 1 -- start start 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.896+0000 7efffb558700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff41045a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.896+0000 7efffb558700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efff419c490 con 0x7efff4105ef0 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.896+0000 7efff92f4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff41045a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.896+0000 7efff92f4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff41045a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56582/0 (socket says 172.21.15.93:56582) 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.896+0000 7efff92f4700 1 -- 172.21.15.93:0/2583423112 learned_addr learned my addr 172.21.15.93:0/2583423112 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.897+0000 7efff92f4700 1 -- 172.21.15.93:0/2583423112 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe4009550 con 0x7efff4105ef0 2023-12-14T18:19:24.562 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.897+0000 7efff92f4700 1 --2- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff41045a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7effe4009fb0 tx=0x7effe4011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.897+0000 7effea7fc700 1 -- 172.21.15.93:0/2583423112 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe4003930 con 0x7efff4105ef0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.897+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efff4108d90 con 0x7efff4105ef0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.897+0000 7effea7fc700 1 -- 172.21.15.93:0/2583423112 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1074+0+0 (secure 0 0 0) 0x7effe4003a90 con 0x7efff4105ef0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.897+0000 7effea7fc700 1 -- 172.21.15.93:0/2583423112 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7effe40195e0 con 0x7efff4105ef0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.897+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efff41059b0 con 0x7efff4105ef0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.898+0000 7effea7fc700 1 -- 172.21.15.93:0/2583423112 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7effe40198a0 con 0x7efff4105ef0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.898+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efff4062120 con 0x7efff4105ef0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.898+0000 7effea7fc700 1 --2- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7effe00386c0 0x7effe003ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.898+0000 7effea7fc700 1 -- 172.21.15.93:0/2583423112 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7effe404c9f0 con 0x7efff4105ef0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.898+0000 7efff8af3700 1 --2- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7effe00386c0 0x7effe003ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.899+0000 7efff8af3700 1 --2- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7effe00386c0 0x7effe003ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efff0006fd0 tx=0x7efff0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:23.903+0000 7effea7fc700 1 -- 172.21.15.93:0/2583423112 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7effe402b340 con 0x7efff4105ef0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.113+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7efff4100b60 con 0x7effe00386c0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.117+0000 7effea7fc700 1 -- 172.21.15.93:0/2583423112 <== mgr.14120 v2:172.21.15.93:6800/2647964405 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7efff4100b60 con 0x7effe00386c0 2023-12-14T18:19:24.563 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7effe00386c0 msgr2=0x7effe003ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 --2- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7effe00386c0 0x7effe003ab70 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7efff0006fd0 tx=0x7efff0006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 msgr2=0x7efff41045a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 --2- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff41045a0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7effe4009fb0 tx=0x7effe4011ad0 comp rx=0 tx=0).stop 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 shutdown_connections 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 --2- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7effe00386c0 0x7effe003ab70 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 --2- 172.21.15.93:0/2583423112 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efff4105ef0 0x7efff41045a0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 >> 172.21.15.93:0/2583423112 conn(0x7efff40ff650 msgr2=0x7efff4100450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 shutdown_connections 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:24.119+0000 7efffb558700 1 -- 172.21.15.93:0/2583423112 wait complete. 2023-12-14T18:19:24.564 INFO:teuthology.orchestra.run.smithi093.stdout:Deploying unmanaged mgr service... 2023-12-14T18:19:24.737 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:24 smithi093 ceph-mon[42070]: Added host smithi093 2023-12-14T18:19:24.737 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:24 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:25.648 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:25 smithi093 ceph-mon[42070]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:25.648 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:25 smithi093 ceph-mon[42070]: Saving service mon spec with placement count:5 2023-12-14T18:19:25.648 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:25 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:25.648 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:25 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:25.648 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:25 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.076+0000 7f7ab204c700 1 Processor -- start 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.076+0000 7f7ab204c700 1 -- start start 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.076+0000 7f7ab204c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac104710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.076+0000 7f7ab204c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac079de0 con 0x7f7aac104300 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.076+0000 7f7aab7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac104710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.077+0000 7f7aab7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac104710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56584/0 (socket says 172.21.15.93:56584) 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.077+0000 7f7aab7fe700 1 -- 172.21.15.93:0/3754614627 learned_addr learned my addr 172.21.15.93:0/3754614627 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.077+0000 7f7aab7fe700 1 -- 172.21.15.93:0/3754614627 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7aac104c50 con 0x7f7aac104300 2023-12-14T18:19:25.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.077+0000 7f7aab7fe700 1 --2- 172.21.15.93:0/3754614627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac104710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0098a0 tx=0x7f7a9c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a16fcd40605bc748 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.077+0000 7f7aaa7fc700 1 -- 172.21.15.93:0/3754614627 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a9c004160 con 0x7f7aac104300 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.077+0000 7f7aaa7fc700 1 -- 172.21.15.93:0/3754614627 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c0042c0 con 0x7f7aac104300 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.077+0000 7f7aaa7fc700 1 -- 172.21.15.93:0/3754614627 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a9c0119b0 con 0x7f7aac104300 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.078+0000 7f7ab204c700 1 -- 172.21.15.93:0/3754614627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 msgr2=0x7f7aac104710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.078+0000 7f7ab204c700 1 --2- 172.21.15.93:0/3754614627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac104710 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c0098a0 tx=0x7f7a9c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.078+0000 7f7ab204c700 1 -- 172.21.15.93:0/3754614627 shutdown_connections 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.078+0000 7f7ab204c700 1 --2- 172.21.15.93:0/3754614627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac104710 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.078+0000 7f7ab204c700 1 -- 172.21.15.93:0/3754614627 >> 172.21.15.93:0/3754614627 conn(0x7f7aac0ffed0 msgr2=0x7f7aac102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.078+0000 7f7ab204c700 1 -- 172.21.15.93:0/3754614627 shutdown_connections 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.078+0000 7f7ab204c700 1 -- 172.21.15.93:0/3754614627 wait complete. 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.079+0000 7f7ab204c700 1 Processor -- start 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.079+0000 7f7ab204c700 1 -- start start 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.079+0000 7f7ab204c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac19b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:25.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.079+0000 7f7ab204c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7aac19bf00 con 0x7f7aac104300 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.079+0000 7f7aab7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac19b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.079+0000 7f7aab7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac19b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56588/0 (socket says 172.21.15.93:56588) 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.079+0000 7f7aab7fe700 1 -- 172.21.15.93:0/3571871431 learned_addr learned my addr 172.21.15.93:0/3571871431 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.079+0000 7f7aab7fe700 1 -- 172.21.15.93:0/3571871431 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c009550 con 0x7f7aac104300 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.079+0000 7f7aab7fe700 1 --2- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac19b9c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c003e60 tx=0x7f7a9c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.080+0000 7f7aa8ff9700 1 -- 172.21.15.93:0/3571871431 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a9c003970 con 0x7f7aac104300 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.080+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7aac105c90 con 0x7f7aac104300 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.080+0000 7f7aa8ff9700 1 -- 172.21.15.93:0/3571871431 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a9c003ad0 con 0x7f7aac104300 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.080+0000 7f7aa8ff9700 1 -- 172.21.15.93:0/3571871431 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a9c0195e0 con 0x7f7aac104300 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.080+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7aac105b00 con 0x7f7aac104300 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.081+0000 7f7aa8ff9700 1 -- 172.21.15.93:0/3571871431 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f7a9c0198a0 con 0x7f7aac104300 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.081+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7aac105e20 con 0x7f7aac104300 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.081+0000 7f7aa8ff9700 1 --2- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f7a940386c0 0x7f7a9403ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:25.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.081+0000 7f7aa8ff9700 1 -- 172.21.15.93:0/3571871431 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f7a9c04c860 con 0x7f7aac104300 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.081+0000 7f7aaaffd700 1 --2- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f7a940386c0 0x7f7a9403ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.081+0000 7f7aaaffd700 1 --2- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f7a940386c0 0x7f7a9403ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0006fd0 tx=0x7f7aa0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.089+0000 7f7aa8ff9700 1 -- 172.21.15.93:0/3571871431 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7a9c019d80 con 0x7f7aac104300 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.313+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f7aac060f30 con 0x7f7a940386c0 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.317+0000 7f7aa8ff9700 1 -- 172.21.15.93:0/3571871431 <== mgr.14120 v2:172.21.15.93:6800/2647964405 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f7aac060f30 con 0x7f7a940386c0 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f7a940386c0 msgr2=0x7f7a9403ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 --2- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f7a940386c0 0x7f7a9403ab70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0006fd0 tx=0x7f7aa0006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 msgr2=0x7f7aac19b9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 --2- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac19b9c0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f7a9c003e60 tx=0x7f7a9c011aa0 comp rx=0 tx=0).stop 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 shutdown_connections 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 --2- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f7a940386c0 0x7f7a9403ab70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 --2- 172.21.15.93:0/3571871431 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7aac104300 0x7f7aac19b9c0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 >> 172.21.15.93:0/3571871431 conn(0x7f7aac0ffed0 msgr2=0x7f7aac18fec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 shutdown_connections 2023-12-14T18:19:25.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:25.320+0000 7f7ab204c700 1 -- 172.21.15.93:0/3571871431 wait complete. 2023-12-14T18:19:26.642 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:26 smithi093 ceph-mon[42070]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:26.642 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:26 smithi093 ceph-mon[42070]: Saving service mgr spec with placement count:2 2023-12-14T18:19:26.642 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:26 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:26.642 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:26 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3860725882' entity='client.admin' 2023-12-14T18:19:26.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.130+0000 7f097170d700 1 Processor -- start 2023-12-14T18:19:26.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.130+0000 7f097170d700 1 -- start start 2023-12-14T18:19:26.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.130+0000 7f097170d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c104d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:26.711 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.130+0000 7f097170d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c105300 con 0x7f096c104920 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.130+0000 7f096affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c104d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.130+0000 7f096affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c104d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56600/0 (socket says 172.21.15.93:56600) 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.130+0000 7f096affd700 1 -- 172.21.15.93:0/959450413 learned_addr learned my addr 172.21.15.93:0/959450413 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.131+0000 7f096affd700 1 -- 172.21.15.93:0/959450413 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c105b20 con 0x7f096c104920 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.131+0000 7f096affd700 1 --2- 172.21.15.93:0/959450413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c104d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098a0 tx=0x7f095c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=852dcfd270991e80 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.131+0000 7f0969ffb700 1 -- 172.21.15.93:0/959450413 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c004160 con 0x7f096c104920 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.131+0000 7f0969ffb700 1 -- 172.21.15.93:0/959450413 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095c0042c0 con 0x7f096c104920 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.131+0000 7f0969ffb700 1 -- 172.21.15.93:0/959450413 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c0119b0 con 0x7f096c104920 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.132+0000 7f097170d700 1 -- 172.21.15.93:0/959450413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 msgr2=0x7f096c104d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.132+0000 7f097170d700 1 --2- 172.21.15.93:0/959450413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c104d30 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f095c0098a0 tx=0x7f095c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.132+0000 7f097170d700 1 -- 172.21.15.93:0/959450413 shutdown_connections 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.132+0000 7f097170d700 1 --2- 172.21.15.93:0/959450413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c104d30 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.132+0000 7f097170d700 1 -- 172.21.15.93:0/959450413 >> 172.21.15.93:0/959450413 conn(0x7f096c0ffed0 msgr2=0x7f096c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:26.712 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.132+0000 7f097170d700 1 -- 172.21.15.93:0/959450413 shutdown_connections 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.132+0000 7f097170d700 1 -- 172.21.15.93:0/959450413 wait complete. 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.132+0000 7f097170d700 1 Processor -- start 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.132+0000 7f097170d700 1 -- start start 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f097170d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c19b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f097170d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c19bef0 con 0x7f096c104920 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f096affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c19b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f096affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c19b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56604/0 (socket says 172.21.15.93:56604) 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f096affd700 1 -- 172.21.15.93:0/3860725882 learned_addr learned my addr 172.21.15.93:0/3860725882 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f096affd700 1 -- 172.21.15.93:0/3860725882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f095c009550 con 0x7f096c104920 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f096affd700 1 --2- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c19b9b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f095c000c00 tx=0x7f095c011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f0953fff700 1 -- 172.21.15.93:0/3860725882 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c0039f0 con 0x7f096c104920 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f0953fff700 1 -- 172.21.15.93:0/3860725882 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f095c003b50 con 0x7f096c104920 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f0953fff700 1 -- 172.21.15.93:0/3860725882 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f095c01a780 con 0x7f096c104920 2023-12-14T18:19:26.713 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c108ee0 con 0x7f096c104920 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.133+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c108d50 con 0x7f096c104920 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.134+0000 7f0953fff700 1 -- 172.21.15.93:0/3860725882 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f095c01a9f0 con 0x7f096c104920 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.134+0000 7f0953fff700 1 --2- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0954038760 0x7f095403ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.134+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c062190 con 0x7f096c104920 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.135+0000 7f0953fff700 1 -- 172.21.15.93:0/3860725882 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f095c04c8f0 con 0x7f096c104920 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.135+0000 7f096a7fc700 1 --2- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0954038760 0x7f095403ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.135+0000 7f096a7fc700 1 --2- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0954038760 0x7f095403ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0960006fd0 tx=0x7f0960006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.139+0000 7f0953fff700 1 -- 172.21.15.93:0/3860725882 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f096c062190 con 0x7f096c104920 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.335+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7f096c062120 con 0x7f096c104920 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.339+0000 7f0953fff700 1 -- 172.21.15.93:0/3860725882 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7f095c0237b0 con 0x7f096c104920 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0954038760 msgr2=0x7f095403ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 --2- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0954038760 0x7f095403ac10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f0960006fd0 tx=0x7f0960006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 msgr2=0x7f096c19b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 --2- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c19b9b0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f095c000c00 tx=0x7f095c011a30 comp rx=0 tx=0).stop 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 shutdown_connections 2023-12-14T18:19:26.714 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 --2- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0954038760 0x7f095403ac10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:26.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 --2- 172.21.15.93:0/3860725882 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f096c104920 0x7f096c19b9b0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:26.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 >> 172.21.15.93:0/3860725882 conn(0x7f096c0ffed0 msgr2=0x7f096c079910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:26.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 shutdown_connections 2023-12-14T18:19:26.715 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:26.341+0000 7f097170d700 1 -- 172.21.15.93:0/3860725882 wait complete. 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.103+0000 7f0a593ad700 1 Processor -- start 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.103+0000 7f0a593ad700 1 -- start start 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a593ad700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a593ad700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54079e60 con 0x7f0a54106490 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a52ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a52ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56610/0 (socket says 172.21.15.93:56610) 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a52ffd700 1 -- 172.21.15.93:0/650509987 learned_addr learned my addr 172.21.15.93:0/650509987 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a52ffd700 1 -- 172.21.15.93:0/650509987 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a54108db0 con 0x7f0a54106490 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a52ffd700 1 --2- 172.21.15.93:0/650509987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54108870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a440098a0 tx=0x7f0a44009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b30d57d395cf627d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a51ffb700 1 -- 172.21.15.93:0/650509987 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a44004160 con 0x7f0a54106490 2023-12-14T18:19:27.687 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a51ffb700 1 -- 172.21.15.93:0/650509987 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a440042c0 con 0x7f0a54106490 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.104+0000 7f0a51ffb700 1 -- 172.21.15.93:0/650509987 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a440119b0 con 0x7f0a54106490 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.105+0000 7f0a593ad700 1 -- 172.21.15.93:0/650509987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 msgr2=0x7f0a54108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.105+0000 7f0a593ad700 1 --2- 172.21.15.93:0/650509987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54108870 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f0a440098a0 tx=0x7f0a44009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.105+0000 7f0a593ad700 1 -- 172.21.15.93:0/650509987 shutdown_connections 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.105+0000 7f0a593ad700 1 --2- 172.21.15.93:0/650509987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54108870 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.105+0000 7f0a593ad700 1 -- 172.21.15.93:0/650509987 >> 172.21.15.93:0/650509987 conn(0x7f0a540ffed0 msgr2=0x7f0a54102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.105+0000 7f0a593ad700 1 -- 172.21.15.93:0/650509987 shutdown_connections 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.105+0000 7f0a593ad700 1 -- 172.21.15.93:0/650509987 wait complete. 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.106+0000 7f0a593ad700 1 Processor -- start 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.106+0000 7f0a593ad700 1 -- start start 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.106+0000 7f0a593ad700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54105df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.106+0000 7f0a593ad700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a54106330 con 0x7f0a54106490 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.106+0000 7f0a52ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54105df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:27.688 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.106+0000 7f0a52ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54105df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56624/0 (socket says 172.21.15.93:56624) 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.106+0000 7f0a52ffd700 1 -- 172.21.15.93:0/3880527082 learned_addr learned my addr 172.21.15.93:0/3880527082 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.106+0000 7f0a52ffd700 1 -- 172.21.15.93:0/3880527082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a44009550 con 0x7f0a54106490 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.107+0000 7f0a52ffd700 1 --2- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54105df0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a44011980 tx=0x7f0a44011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.107+0000 7f0a3bfff700 1 -- 172.21.15.93:0/3880527082 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a44003a60 con 0x7f0a54106490 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.107+0000 7f0a3bfff700 1 -- 172.21.15.93:0/3880527082 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0a44003bc0 con 0x7f0a54106490 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.107+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a54109d80 con 0x7f0a54106490 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.107+0000 7f0a3bfff700 1 -- 172.21.15.93:0/3880527082 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0a4401a670 con 0x7f0a54106490 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.107+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a54109bf0 con 0x7f0a54106490 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.108+0000 7f0a3bfff700 1 -- 172.21.15.93:0/3880527082 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f0a4401a930 con 0x7f0a54106490 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.108+0000 7f0a3bfff700 1 --2- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0a3c038710 0x7f0a3c03abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.108+0000 7f0a3bfff700 1 -- 172.21.15.93:0/3880527082 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f0a4404d6a0 con 0x7f0a54106490 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.108+0000 7f0a527fc700 1 --2- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0a3c038710 0x7f0a3c03abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:27.689 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.108+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a54062120 con 0x7f0a54106490 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.108+0000 7f0a527fc700 1 --2- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0a3c038710 0x7f0a3c03abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0a48006fd0 tx=0x7f0a48006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.113+0000 7f0a3bfff700 1 -- 172.21.15.93:0/3880527082 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0a4401abe0 con 0x7f0a54106490 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.330+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f0a54104ba0 con 0x7f0a54106490 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.333+0000 7f0a3bfff700 1 -- 172.21.15.93:0/3880527082 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f0a44018350 con 0x7f0a54106490 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0a3c038710 msgr2=0x7f0a3c03abc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 --2- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0a3c038710 0x7f0a3c03abc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0a48006fd0 tx=0x7f0a48006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 msgr2=0x7f0a54105df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 --2- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54105df0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f0a44011980 tx=0x7f0a44011a60 comp rx=0 tx=0).stop 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 shutdown_connections 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 --2- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f0a3c038710 0x7f0a3c03abc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 --2- 172.21.15.93:0/3880527082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0a54106490 0x7f0a54105df0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 >> 172.21.15.93:0/3880527082 conn(0x7f0a540ffed0 msgr2=0x7f0a54100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 shutdown_connections 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:27.336+0000 7f0a593ad700 1 -- 172.21.15.93:0/3880527082 wait complete. 2023-12-14T18:19:27.690 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling the dashboard module... 2023-12-14T18:19:28.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:28 smithi093 ceph-mon[42070]: from='mgr.14120 172.21.15.93:0/3158114637' entity='mgr.x' 2023-12-14T18:19:28.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:28 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3880527082' entity='client.admin' 2023-12-14T18:19:29.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:29 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3718667907' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2023-12-14T18:19:29.565 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:29 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: ignoring --setuser ceph since I am not root 2023-12-14T18:19:29.565 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:29 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: ignoring --setgroup ceph since I am not root 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.022+0000 7f4bb34bd700 1 Processor -- start 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.022+0000 7f4bb34bd700 1 -- start start 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.022+0000 7f4bb34bd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac07a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.022+0000 7f4bb34bd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac07a7d0 con 0x7f4bac07bd90 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.023+0000 7f4bb1259700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac07a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.023+0000 7f4bb1259700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac07a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56626/0 (socket says 172.21.15.93:56626) 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.023+0000 7f4bb1259700 1 -- 172.21.15.93:0/4284841787 learned_addr learned my addr 172.21.15.93:0/4284841787 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.023+0000 7f4bb1259700 1 -- 172.21.15.93:0/4284841787 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bac07a910 con 0x7f4bac07bd90 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.023+0000 7f4bb1259700 1 --2- 172.21.15.93:0/4284841787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac07a290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009e80 tx=0x7f4ba400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5122c055bc78f4af server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.023+0000 7f4ba3fff700 1 -- 172.21.15.93:0/4284841787 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ba400e070 con 0x7f4bac07bd90 2023-12-14T18:19:29.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.023+0000 7f4ba3fff700 1 -- 172.21.15.93:0/4284841787 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ba400b8c0 con 0x7f4bac07bd90 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.023+0000 7f4ba3fff700 1 -- 172.21.15.93:0/4284841787 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ba40043e0 con 0x7f4bac07bd90 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.024+0000 7f4bb34bd700 1 -- 172.21.15.93:0/4284841787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 msgr2=0x7f4bac07a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.024+0000 7f4bb34bd700 1 --2- 172.21.15.93:0/4284841787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac07a290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009e80 tx=0x7f4ba400b330 comp rx=0 tx=0).stop 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.024+0000 7f4bb34bd700 1 -- 172.21.15.93:0/4284841787 shutdown_connections 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.024+0000 7f4bb34bd700 1 --2- 172.21.15.93:0/4284841787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac07a290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.024+0000 7f4bb34bd700 1 -- 172.21.15.93:0/4284841787 >> 172.21.15.93:0/4284841787 conn(0x7f4bac101170 msgr2=0x7f4bac103580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.024+0000 7f4bb34bd700 1 -- 172.21.15.93:0/4284841787 shutdown_connections 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.024+0000 7f4bb34bd700 1 -- 172.21.15.93:0/4284841787 wait complete. 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb34bd700 1 Processor -- start 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb34bd700 1 -- start start 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb34bd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac19ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb34bd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bac19bf80 con 0x7f4bac07bd90 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb1259700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac19ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb1259700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac19ba40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56640/0 (socket says 172.21.15.93:56640) 2023-12-14T18:19:29.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb1259700 1 -- 172.21.15.93:0/3718667907 learned_addr learned my addr 172.21.15.93:0/3718667907 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb1259700 1 -- 172.21.15.93:0/3718667907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ba4009b30 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb1259700 1 --2- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac19ba40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009530 tx=0x7f4ba4003d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4ba27fc700 1 -- 172.21.15.93:0/3718667907 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ba400e040 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4ba27fc700 1 -- 172.21.15.93:0/3718667907 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ba400bd70 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bac10c000 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4ba27fc700 1 -- 172.21.15.93:0/3718667907 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f4ba401c700 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.025+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bac10be70 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.026+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bac10c190 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.027+0000 7f4ba27fc700 1 -- 172.21.15.93:0/3718667907 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 8) v1 ==== 45483+0+0 (secure 0 0 0) 0x7f4ba4012a80 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.027+0000 7f4ba27fc700 1 --2- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f4b98038760 0x7f4b9803ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.027+0000 7f4ba27fc700 1 -- 172.21.15.93:0/3718667907 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f4ba404c6d0 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.027+0000 7f4bb0a58700 1 --2- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f4b98038760 0x7f4b9803ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.028+0000 7f4bb0a58700 1 --2- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f4b98038760 0x7f4b9803ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c006fd0 tx=0x7f4b9c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.035+0000 7f4ba27fc700 1 -- 172.21.15.93:0/3718667907 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4ba4012430 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:28.297+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f4bac19c5a0 con 0x7f4bac07bd90 2023-12-14T18:19:29.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.264+0000 7f4ba27fc700 1 -- 172.21.15.93:0/3718667907 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f4ba404b850 con 0x7f4bac07bd90 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.265+0000 7f4ba27fc700 1 -- 172.21.15.93:0/3718667907 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f4ba404bbd0 con 0x7f4bac07bd90 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f4b98038760 msgr2=0x7f4b9803ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 --2- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f4b98038760 0x7f4b9803ac10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4b9c006fd0 tx=0x7f4b9c006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 msgr2=0x7f4bac19ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 --2- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac19ba40 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009530 tx=0x7f4ba4003d00 comp rx=0 tx=0).stop 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 shutdown_connections 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 --2- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f4b98038760 0x7f4b9803ac10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 --2- 172.21.15.93:0/3718667907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4bac07bd90 0x7f4bac19ba40 secure :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4ba4009530 tx=0x7f4ba4003d00 comp rx=0 tx=0).stop 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 >> 172.21.15.93:0/3718667907 conn(0x7f4bac101170 msgr2=0x7f4bac101f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 shutdown_connections 2023-12-14T18:19:29.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:29.268+0000 7f4bb34bd700 1 -- 172.21.15.93:0/3718667907 wait complete. 2023-12-14T18:19:29.864 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:29 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:29.654+0000 7fdae8385200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:19:29.864 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:29 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:29.765+0000 7fdae8385200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:19:30.264 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:29 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:29.862+0000 7fdae8385200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:19:30.264 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:29 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:29.951+0000 7fdae8385200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:19:30.520 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:30 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3718667907' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2023-12-14T18:19:30.521 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:30 smithi093 ceph-mon[42070]: mgrmap e9: x(active, since 14s) 2023-12-14T18:19:30.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "epoch": 9, 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "available": true, 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "active_name": "x", 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "num_standby": 0 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.014+0000 7f3cc071f700 1 Processor -- start 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.014+0000 7f3cc071f700 1 -- start start 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.015+0000 7f3cc071f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb8104730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.015+0000 7f3cc071f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb8079dc0 con 0x7f3cb8104320 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.015+0000 7f3cbe4bb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb8104730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.015+0000 7f3cbe4bb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb8104730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56664/0 (socket says 172.21.15.93:56664) 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.015+0000 7f3cbe4bb700 1 -- 172.21.15.93:0/577230051 learned_addr learned my addr 172.21.15.93:0/577230051 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:30.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.015+0000 7f3cbe4bb700 1 -- 172.21.15.93:0/577230051 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb8079f00 con 0x7f3cb8104320 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.015+0000 7f3cbe4bb700 1 --2- 172.21.15.93:0/577230051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb8104730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80098a0 tx=0x7f3ca8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615fc3cb26e125ce server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cbd4b9700 1 -- 172.21.15.93:0/577230051 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ca8004160 con 0x7f3cb8104320 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cbd4b9700 1 -- 172.21.15.93:0/577230051 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca80042c0 con 0x7f3cb8104320 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cbd4b9700 1 -- 172.21.15.93:0/577230051 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ca80119b0 con 0x7f3cb8104320 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cc071f700 1 -- 172.21.15.93:0/577230051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 msgr2=0x7f3cb8104730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cc071f700 1 --2- 172.21.15.93:0/577230051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb8104730 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f3ca80098a0 tx=0x7f3ca8009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cc071f700 1 -- 172.21.15.93:0/577230051 shutdown_connections 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cc071f700 1 --2- 172.21.15.93:0/577230051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb8104730 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cc071f700 1 -- 172.21.15.93:0/577230051 >> 172.21.15.93:0/577230051 conn(0x7f3cb80ffed0 msgr2=0x7f3cb8102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cc071f700 1 -- 172.21.15.93:0/577230051 shutdown_connections 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.016+0000 7f3cc071f700 1 -- 172.21.15.93:0/577230051 wait complete. 2023-12-14T18:19:30.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.017+0000 7f3cc071f700 1 Processor -- start 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.017+0000 7f3cc071f700 1 -- start start 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.017+0000 7f3cc071f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb819b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.017+0000 7f3cc071f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb819bed0 con 0x7f3cb8104320 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.017+0000 7f3cbe4bb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb819b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.017+0000 7f3cbe4bb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb819b990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56666/0 (socket says 172.21.15.93:56666) 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.017+0000 7f3cbe4bb700 1 -- 172.21.15.93:0/4103637550 learned_addr learned my addr 172.21.15.93:0/4103637550 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.017+0000 7f3cbe4bb700 1 -- 172.21.15.93:0/4103637550 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca8009550 con 0x7f3cb8104320 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.018+0000 7f3cbe4bb700 1 --2- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb819b990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003e60 tx=0x7f3ca8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.018+0000 7f3caf7fe700 1 -- 172.21.15.93:0/4103637550 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ca8003970 con 0x7f3cb8104320 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.018+0000 7f3caf7fe700 1 -- 172.21.15.93:0/4103637550 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3ca8003ad0 con 0x7f3cb8104320 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.018+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb8105b20 con 0x7f3cb8104320 2023-12-14T18:19:30.630 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.018+0000 7f3caf7fe700 1 -- 172.21.15.93:0/4103637550 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3ca80195e0 con 0x7f3cb8104320 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.018+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb8105990 con 0x7f3cb8104320 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.019+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb8105cb0 con 0x7f3cb8104320 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.019+0000 7f3caf7fe700 1 -- 172.21.15.93:0/4103637550 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f3ca801a430 con 0x7f3cb8104320 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.019+0000 7f3caf7fe700 1 --2- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f3ca4038710 0x7f3ca403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.019+0000 7f3caf7fe700 1 -- 172.21.15.93:0/4103637550 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7f3ca804c8d0 con 0x7f3cb8104320 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.019+0000 7f3cbdcba700 1 -- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f3ca4038710 msgr2=0x7f3ca403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2647964405 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.019+0000 7f3cbdcba700 1 --2- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f3ca4038710 0x7f3ca403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.024+0000 7f3caf7fe700 1 -- 172.21.15.93:0/4103637550 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3ca8023af0 con 0x7f3cb8104320 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.220+0000 7f3cbdcba700 1 -- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f3ca4038710 msgr2=0x7f3ca403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2647964405 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.220+0000 7f3cbdcba700 1 --2- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f3ca4038710 0x7f3ca403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.276+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f3cb802cfe0 con 0x7f3cb8104320 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.277+0000 7f3caf7fe700 1 -- 172.21.15.93:0/4103637550 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f3ca8023af0 con 0x7f3cb8104320 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f3ca4038710 msgr2=0x7f3ca403abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 --2- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f3ca4038710 0x7f3ca403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:30.631 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 msgr2=0x7f3cb819b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:30.632 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 --2- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb819b990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3ca8003e60 tx=0x7f3ca8011aa0 comp rx=0 tx=0).stop 2023-12-14T18:19:30.632 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 shutdown_connections 2023-12-14T18:19:30.632 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 --2- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7f3ca4038710 0x7f3ca403abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:30.632 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 --2- 172.21.15.93:0/4103637550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3cb8104320 0x7f3cb819b990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:30.632 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 >> 172.21.15.93:0/4103637550 conn(0x7f3cb80ffed0 msgr2=0x7f3cb818fee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:30.632 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 shutdown_connections 2023-12-14T18:19:30.632 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.279+0000 7f3cc071f700 1 -- 172.21.15.93:0/4103637550 wait complete. 2023-12-14T18:19:30.632 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for the mgr to restart... 2023-12-14T18:19:30.632 INFO:teuthology.orchestra.run.smithi093.stdout:Waiting for mgr epoch 9... 2023-12-14T18:19:30.796 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:30 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:30.581+0000 7fdae8385200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:19:31.126 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:30 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:30.828+0000 7fdae8385200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:19:31.126 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:30 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:30.918+0000 7fdae8385200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:19:31.126 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:31 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:31.033+0000 7fdae8385200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:19:31.126 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:31 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:31.124+0000 7fdae8385200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:19:31.407 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:31 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4103637550' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2023-12-14T18:19:31.407 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:31 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:31.319+0000 7fdae8385200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:19:31.778 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:31 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:31.405+0000 7fdae8385200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:19:32.042 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:31 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:31.776+0000 7fdae8385200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:19:32.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:32 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:32.041+0000 7fdae8385200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:19:32.814 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:32 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:32.321+0000 7fdae8385200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:19:32.814 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:32 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:32.415+0000 7fdae8385200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:19:33.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:33 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:33.162+0000 7fdae8385200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:19:33.565 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:33 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:33.258+0000 7fdae8385200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:19:33.565 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:33 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:33.406+0000 7fdae8385200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:19:34.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:34 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:34.034+0000 7fdae8385200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:19:34.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:34 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:34.216+0000 7fdae8385200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:19:34.814 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:34 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:34.411+0000 7fdae8385200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:19:35.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:35 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:35.118+0000 7fdae8385200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:19:35.872 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:35 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:35.590+0000 7fdae8385200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:19:35.872 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:35 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:35.679+0000 7fdae8385200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:19:36.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:19:35 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:19:35.870+0000 7fdae8385200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: Active manager daemon x restarted 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: Activating manager daemon x 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: osdmap e3: 0 total, 0 up, 0 in 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: mgrmap e10: x(active, starting, since 0.00321605s) 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: Manager daemon x is now available 2023-12-14T18:19:36.927 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:36.928 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:36.928 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:19:36.928 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout { 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout "initialized": true 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout } 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.997+0000 7fa32174f700 1 Processor -- start 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.997+0000 7fa32174f700 1 -- start start 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.997+0000 7fa32174f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.997+0000 7fa32174f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c079e60 con 0x7fa31c106490 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.997+0000 7fa31affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.997+0000 7fa31affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56674/0 (socket says 172.21.15.93:56674) 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.997+0000 7fa31affd700 1 -- 172.21.15.93:0/203266189 learned_addr learned my addr 172.21.15.93:0/203266189 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:37.253 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.998+0000 7fa31affd700 1 -- 172.21.15.93:0/203266189 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa31c108db0 con 0x7fa31c106490 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.998+0000 7fa31affd700 1 --2- 172.21.15.93:0/203266189 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c108870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0098a0 tx=0x7fa30c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609ecb9d301a1f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.998+0000 7fa319ffb700 1 -- 172.21.15.93:0/203266189 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa30c004160 con 0x7fa31c106490 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.998+0000 7fa319ffb700 1 -- 172.21.15.93:0/203266189 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30c0042c0 con 0x7fa31c106490 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.998+0000 7fa319ffb700 1 -- 172.21.15.93:0/203266189 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa30c0119b0 con 0x7fa31c106490 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.999+0000 7fa32174f700 1 -- 172.21.15.93:0/203266189 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 msgr2=0x7fa31c108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.999+0000 7fa32174f700 1 --2- 172.21.15.93:0/203266189 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c108870 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa30c0098a0 tx=0x7fa30c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.999+0000 7fa32174f700 1 -- 172.21.15.93:0/203266189 shutdown_connections 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.999+0000 7fa32174f700 1 --2- 172.21.15.93:0/203266189 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c108870 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.999+0000 7fa32174f700 1 -- 172.21.15.93:0/203266189 >> 172.21.15.93:0/203266189 conn(0x7fa31c0ffed0 msgr2=0x7fa31c102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.999+0000 7fa32174f700 1 -- 172.21.15.93:0/203266189 shutdown_connections 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.999+0000 7fa32174f700 1 -- 172.21.15.93:0/203266189 wait complete. 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:30.999+0000 7fa32174f700 1 Processor -- start 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.000+0000 7fa32174f700 1 -- start start 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.000+0000 7fa32174f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.000+0000 7fa32174f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa31c199cc0 con 0x7fa31c106490 2023-12-14T18:19:37.254 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.000+0000 7fa31affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.000+0000 7fa31affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56686/0 (socket says 172.21.15.93:56686) 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.000+0000 7fa31affd700 1 -- 172.21.15.93:0/3119595624 learned_addr learned my addr 172.21.15.93:0/3119595624 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.000+0000 7fa31affd700 1 -- 172.21.15.93:0/3119595624 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa30c009550 con 0x7fa31c106490 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.000+0000 7fa31affd700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c199780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa30c003e60 tx=0x7fa30c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.000+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa30c003970 con 0x7fa31c106490 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.001+0000 7fa32174f700 1 -- 172.21.15.93:0/3119595624 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa31c109d80 con 0x7fa31c106490 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.001+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa30c003ad0 con 0x7fa31c106490 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.001+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa30c0195e0 con 0x7fa31c106490 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.001+0000 7fa32174f700 1 -- 172.21.15.93:0/3119595624 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa31c109bf0 con 0x7fa31c106490 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.002+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 9) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fa30c0198e0 con 0x7fa31c106490 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.002+0000 7fa303fff700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 0x7fa30403abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.002+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 --> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa30403b2d0 con 0x7fa304038710 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.002+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 940+0+0 (secure 0 0 0) 0x7fa30c04c8b0 con 0x7fa31c106490 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.002+0000 7fa31a7fc700 1 -- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 msgr2=0x7fa30403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2647964405 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.002+0000 7fa31a7fc700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 0x7fa30403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:19:37.255 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.203+0000 7fa31a7fc700 1 -- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 msgr2=0x7fa30403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2647964405 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.203+0000 7fa31a7fc700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 0x7fa30403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.604+0000 7fa31a7fc700 1 -- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 msgr2=0x7fa30403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2647964405 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:31.604+0000 7fa31a7fc700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 0x7fa30403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:32.405+0000 7fa31a7fc700 1 -- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 msgr2=0x7fa30403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2647964405 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:32.405+0000 7fa31a7fc700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 0x7fa30403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:34.006+0000 7fa31a7fc700 1 -- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 msgr2=0x7fa30403abc0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/2647964405 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:34.006+0000 7fa31a7fc700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 0x7fa30403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:35.885+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mgrmap(e 10) v1 ==== 45263+0+0 (secure 0 0 0) 0x7fa30c019be0 con 0x7fa31c106490 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:35.885+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 msgr2=0x7fa30403abc0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:35.885+0000 7fa303fff700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 0x7fa30403abc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.886+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7fa30c023390 con 0x7fa31c106490 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.886+0000 7fa303fff700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa30403bff0 0x7fa30403e3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.886+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa30403b2d0 con 0x7fa30403bff0 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.887+0000 7fa31a7fc700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa30403bff0 0x7fa30403e3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.888+0000 7fa31a7fc700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa30403bff0 0x7fa30403e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa310003a10 tx=0x7fa3100092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:37.256 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.888+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6910 (secure 0 0 0) 0x7fa30403b2d0 con 0x7fa30403bff0 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 -- 172.21.15.93:0/3119595624 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7fa31c19aac0 con 0x7fa30403bff0 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa303fff700 1 -- 172.21.15.93:0/3119595624 <== mgr.14152 v2:172.21.15.93:6800/3802499407 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7fa31c19aac0 con 0x7fa30403bff0 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 -- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa30403bff0 msgr2=0x7fa30403e3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa30403bff0 0x7fa30403e3d0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa310003a10 tx=0x7fa3100092b0 comp rx=0 tx=0).stop 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 -- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 msgr2=0x7fa31c199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c199780 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa30c003e60 tx=0x7fa30c011aa0 comp rx=0 tx=0).stop 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 -- 172.21.15.93:0/3119595624 shutdown_connections 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa30403bff0 0x7fa30403e3d0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa31c106490 0x7fa31c199780 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 --2- 172.21.15.93:0/3119595624 >> [v2:172.21.15.93:6800/2647964405,v1:172.21.15.93:6801/2647964405] conn(0x7fa304038710 0x7fa30403abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.896+0000 7fa32174f700 1 -- 172.21.15.93:0/3119595624 >> 172.21.15.93:0/3119595624 conn(0x7fa31c0ffed0 msgr2=0x7fa31c100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.897+0000 7fa32174f700 1 -- 172.21.15.93:0/3119595624 shutdown_connections 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:36.897+0000 7fa32174f700 1 -- 172.21.15.93:0/3119595624 wait complete. 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:mgr epoch 9 is available 2023-12-14T18:19:37.257 INFO:teuthology.orchestra.run.smithi093.stdout:Generating a dashboard self-signed certificate... 2023-12-14T18:19:38.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:37 smithi093 ceph-mon[42070]: [14/Dec/2023:18:19:36] ENGINE Bus STARTING 2023-12-14T18:19:38.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:37 smithi093 ceph-mon[42070]: [14/Dec/2023:18:19:36] ENGINE Serving on https://172.21.15.93:7150 2023-12-14T18:19:38.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:37 smithi093 ceph-mon[42070]: [14/Dec/2023:18:19:36] ENGINE Serving on http://172.21.15.93:8765 2023-12-14T18:19:38.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:37 smithi093 ceph-mon[42070]: [14/Dec/2023:18:19:36] ENGINE Bus STARTED 2023-12-14T18:19:38.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:38.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:37 smithi093 ceph-mon[42070]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2023-12-14T18:19:38.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:37 smithi093 ceph-mon[42070]: mgrmap e11: x(active, since 1.00584s) 2023-12-14T18:19:38.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:37 smithi093 ceph-mon[42070]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2023-12-14T18:19:38.145 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:38.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2023-12-14T18:19:38.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.666+0000 7ffbc4889700 1 Processor -- start 2023-12-14T18:19:38.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.667+0000 7ffbc4889700 1 -- start start 2023-12-14T18:19:38.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.667+0000 7ffbc4889700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc0104110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:38.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.667+0000 7ffbc4889700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc01046e0 con 0x7ffbc0103d00 2023-12-14T18:19:38.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.667+0000 7ffbbe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc0104110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:38.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.667+0000 7ffbbe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc0104110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41312/0 (socket says 172.21.15.93:41312) 2023-12-14T18:19:38.294 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.667+0000 7ffbbe59c700 1 -- 172.21.15.93:0/1225154864 learned_addr learned my addr 172.21.15.93:0/1225154864 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.667+0000 7ffbbe59c700 1 -- 172.21.15.93:0/1225154864 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc0104820 con 0x7ffbc0103d00 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.667+0000 7ffbbe59c700 1 --2- 172.21.15.93:0/1225154864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc0104110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00098a0 tx=0x7ffbb0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6b1cc7b0b2c4c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.668+0000 7ffbbd59a700 1 -- 172.21.15.93:0/1225154864 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbb0004160 con 0x7ffbc0103d00 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.668+0000 7ffbbd59a700 1 -- 172.21.15.93:0/1225154864 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb00042c0 con 0x7ffbc0103d00 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.668+0000 7ffbbd59a700 1 -- 172.21.15.93:0/1225154864 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbb00119b0 con 0x7ffbc0103d00 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.668+0000 7ffbc4889700 1 -- 172.21.15.93:0/1225154864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 msgr2=0x7ffbc0104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.668+0000 7ffbc4889700 1 --2- 172.21.15.93:0/1225154864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc0104110 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ffbb00098a0 tx=0x7ffbb0009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.669+0000 7ffbc4889700 1 -- 172.21.15.93:0/1225154864 shutdown_connections 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.669+0000 7ffbc4889700 1 --2- 172.21.15.93:0/1225154864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc0104110 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.669+0000 7ffbc4889700 1 -- 172.21.15.93:0/1225154864 >> 172.21.15.93:0/1225154864 conn(0x7ffbc00ff650 msgr2=0x7ffbc0101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.669+0000 7ffbc4889700 1 -- 172.21.15.93:0/1225154864 shutdown_connections 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.669+0000 7ffbc4889700 1 -- 172.21.15.93:0/1225154864 wait complete. 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.669+0000 7ffbc4889700 1 Processor -- start 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.669+0000 7ffbc4889700 1 -- start start 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbc4889700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc01066e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:38.295 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbc4889700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc0106c20 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbbe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc01066e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbbe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc01066e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41328/0 (socket says 172.21.15.93:41328) 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbbe59c700 1 -- 172.21.15.93:0/493550624 learned_addr learned my addr 172.21.15.93:0/493550624 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbbe59c700 1 -- 172.21.15.93:0/493550624 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbb0009550 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbbe59c700 1 --2- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc01066e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0000c00 tx=0x7ffbb0011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbaf7fe700 1 -- 172.21.15.93:0/493550624 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbb00039f0 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbaf7fe700 1 -- 172.21.15.93:0/493550624 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffbb0003b50 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc0107b60 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbaf7fe700 1 -- 172.21.15.93:0/493550624 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffbb001a780 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.670+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc01079d0 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.671+0000 7ffbaf7fe700 1 -- 172.21.15.93:0/493550624 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7ffbb001aa40 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.671+0000 7ffbaf7fe700 1 --2- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffba8038630 0x7ffba803aae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.671+0000 7ffbaf7fe700 1 -- 172.21.15.93:0/493550624 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7ffbb004c710 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.671+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc0062120 con 0x7ffbc0103d00 2023-12-14T18:19:38.296 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.671+0000 7ffbbdd9b700 1 --2- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffba8038630 0x7ffba803aae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.672+0000 7ffbbdd9b700 1 --2- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffba8038630 0x7ffba803aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4006fd0 tx=0x7ffbb4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.676+0000 7ffbaf7fe700 1 -- 172.21.15.93:0/493550624 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffbb001ad40 con 0x7ffbc0103d00 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.883+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbc0105490 con 0x7ffba8038630 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.963+0000 7ffbaf7fe700 1 -- 172.21.15.93:0/493550624 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ffbc0105490 con 0x7ffba8038630 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffba8038630 msgr2=0x7ffba803aae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 --2- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffba8038630 0x7ffba803aae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffbb4006fd0 tx=0x7ffbb4006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 msgr2=0x7ffbc01066e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 --2- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc01066e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffbb0000c00 tx=0x7ffbb0011a30 comp rx=0 tx=0).stop 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 shutdown_connections 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 --2- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffba8038630 0x7ffba803aae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 --2- 172.21.15.93:0/493550624 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffbc0103d00 0x7ffbc01066e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 >> 172.21.15.93:0/493550624 conn(0x7ffbc00ff650 msgr2=0x7ffbc0100d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 shutdown_connections 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:37.966+0000 7ffbc4889700 1 -- 172.21.15.93:0/493550624 wait complete. 2023-12-14T18:19:38.297 INFO:teuthology.orchestra.run.smithi093.stdout:Creating initial admin user... 2023-12-14T18:19:39.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:38 smithi093 ceph-mon[42070]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:39.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:38 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:39.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:38 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:39.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:38 smithi093 ceph-mon[42070]: mgrmap e12: x(active, since 2s) 2023-12-14T18:19:39.759 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Abmwa0j6UqBKi8kxa47OJeazrVWRTOsCJPP8NsHVqQ5L6V4CHLMdK", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1702577979, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.666+0000 7f124aa62700 1 Processor -- start 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.666+0000 7f124aa62700 1 -- start start 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.666+0000 7f124aa62700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f12441063b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.666+0000 7f124aa62700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1244106980 con 0x7f1244105fa0 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.667+0000 7f1243fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f12441063b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.667+0000 7f1243fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f12441063b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41334/0 (socket says 172.21.15.93:41334) 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.667+0000 7f1243fff700 1 -- 172.21.15.93:0/2348618411 learned_addr learned my addr 172.21.15.93:0/2348618411 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.667+0000 7f1243fff700 1 -- 172.21.15.93:0/2348618411 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1244106ac0 con 0x7f1244105fa0 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.667+0000 7f1243fff700 1 --2- 172.21.15.93:0/2348618411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f12441063b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f122c0098a0 tx=0x7f122c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c894935b025d878a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.667+0000 7f1242ffd700 1 -- 172.21.15.93:0/2348618411 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f122c004160 con 0x7f1244105fa0 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.667+0000 7f1242ffd700 1 -- 172.21.15.93:0/2348618411 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122c0042c0 con 0x7f1244105fa0 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.667+0000 7f1242ffd700 1 -- 172.21.15.93:0/2348618411 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f122c0119b0 con 0x7f1244105fa0 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.668+0000 7f124aa62700 1 -- 172.21.15.93:0/2348618411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 msgr2=0x7f12441063b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:39.760 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.668+0000 7f124aa62700 1 --2- 172.21.15.93:0/2348618411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f12441063b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f122c0098a0 tx=0x7f122c009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.668+0000 7f124aa62700 1 -- 172.21.15.93:0/2348618411 shutdown_connections 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.668+0000 7f124aa62700 1 --2- 172.21.15.93:0/2348618411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f12441063b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.668+0000 7f124aa62700 1 -- 172.21.15.93:0/2348618411 >> 172.21.15.93:0/2348618411 conn(0x7f12440ff650 msgr2=0x7f1244101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.668+0000 7f124aa62700 1 -- 172.21.15.93:0/2348618411 shutdown_connections 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.668+0000 7f124aa62700 1 -- 172.21.15.93:0/2348618411 wait complete. 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.669+0000 7f124aa62700 1 Processor -- start 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.669+0000 7f124aa62700 1 -- start start 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.669+0000 7f124aa62700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f124419b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.669+0000 7f124aa62700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f124419be40 con 0x7f1244105fa0 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.669+0000 7f1243fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f124419b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.669+0000 7f1243fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f124419b900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41338/0 (socket says 172.21.15.93:41338) 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.669+0000 7f1243fff700 1 -- 172.21.15.93:0/788610131 learned_addr learned my addr 172.21.15.93:0/788610131 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.670+0000 7f1243fff700 1 -- 172.21.15.93:0/788610131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f122c009550 con 0x7f1244105fa0 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.670+0000 7f1243fff700 1 --2- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f124419b900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f122c003e60 tx=0x7f122c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.670+0000 7f12417fa700 1 -- 172.21.15.93:0/788610131 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f122c003970 con 0x7f1244105fa0 2023-12-14T18:19:39.761 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.670+0000 7f12417fa700 1 -- 172.21.15.93:0/788610131 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f122c003ad0 con 0x7f1244105fa0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.670+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12441048e0 con 0x7f1244105fa0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.670+0000 7f12417fa700 1 -- 172.21.15.93:0/788610131 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f122c019590 con 0x7f1244105fa0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.670+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1244104750 con 0x7f1244105fa0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.671+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1244104a70 con 0x7f1244105fa0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.671+0000 7f12417fa700 1 -- 172.21.15.93:0/788610131 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 11) v1 ==== 45390+0+0 (secure 0 0 0) 0x7f122c019850 con 0x7f1244105fa0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.672+0000 7f12417fa700 1 --2- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f12300385d0 0x7f123003aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.672+0000 7f12417fa700 1 -- 172.21.15.93:0/788610131 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f122c04cc60 con 0x7f1244105fa0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.672+0000 7f12437fe700 1 --2- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f12300385d0 0x7f123003aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.672+0000 7f12437fe700 1 --2- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f12300385d0 0x7f123003aa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1234006fd0 tx=0x7f1234006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.676+0000 7f12417fa700 1 -- 172.21.15.93:0/788610131 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f122c003c40 con 0x7f1244105fa0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.850+0000 7f12417fa700 1 -- 172.21.15.93:0/788610131 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f122c04ba90 con 0x7f1244105fa0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:38.893+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1244061f40 con 0x7f12300385d0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.117+0000 7f12417fa700 1 -- 172.21.15.93:0/788610131 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7f1244061f40 con 0x7f12300385d0 2023-12-14T18:19:39.762 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f12300385d0 msgr2=0x7f123003aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 --2- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f12300385d0 0x7f123003aa80 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1234006fd0 tx=0x7f1234006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 msgr2=0x7f124419b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 --2- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f124419b900 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f122c003e60 tx=0x7f122c011aa0 comp rx=0 tx=0).stop 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 shutdown_connections 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 --2- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f12300385d0 0x7f123003aa80 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 --2- 172.21.15.93:0/788610131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1244105fa0 0x7f124419b900 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 >> 172.21.15.93:0/788610131 conn(0x7f12440ff650 msgr2=0x7f1244192740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 shutdown_connections 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:39.119+0000 7f124aa62700 1 -- 172.21.15.93:0/788610131 wait complete. 2023-12-14T18:19:39.763 INFO:teuthology.orchestra.run.smithi093.stdout:Fetching dashboard port number... 2023-12-14T18:19:40.461 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:40 smithi093 ceph-mon[42070]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:40.461 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:40.661 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stdout 8443 2023-12-14T18:19:40.661 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.102+0000 7f7a69759700 1 Processor -- start 2023-12-14T18:19:40.661 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.102+0000 7f7a69759700 1 -- start start 2023-12-14T18:19:40.661 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.102+0000 7f7a69759700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a64105e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:40.661 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.102+0000 7f7a69759700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a64106400 con 0x7f7a64105a20 2023-12-14T18:19:40.661 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.102+0000 7f7a62ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a64105e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:40.661 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.102+0000 7f7a62ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a64105e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41350/0 (socket says 172.21.15.93:41350) 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.102+0000 7f7a62ffd700 1 -- 172.21.15.93:0/1696336221 learned_addr learned my addr 172.21.15.93:0/1696336221 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.103+0000 7f7a62ffd700 1 -- 172.21.15.93:0/1696336221 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a64106c20 con 0x7f7a64105a20 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.103+0000 7f7a62ffd700 1 --2- 172.21.15.93:0/1696336221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a64105e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098a0 tx=0x7f7a58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b049027a5178786 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.103+0000 7f7a61ffb700 1 -- 172.21.15.93:0/1696336221 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a58004160 con 0x7f7a64105a20 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.103+0000 7f7a61ffb700 1 -- 172.21.15.93:0/1696336221 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a580042c0 con 0x7f7a64105a20 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.103+0000 7f7a61ffb700 1 -- 172.21.15.93:0/1696336221 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a580119b0 con 0x7f7a64105a20 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.103+0000 7f7a69759700 1 -- 172.21.15.93:0/1696336221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 msgr2=0x7f7a64105e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.103+0000 7f7a69759700 1 --2- 172.21.15.93:0/1696336221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a64105e30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f7a580098a0 tx=0x7f7a58009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.103+0000 7f7a69759700 1 -- 172.21.15.93:0/1696336221 shutdown_connections 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.104+0000 7f7a69759700 1 --2- 172.21.15.93:0/1696336221 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a64105e30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:40.662 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.104+0000 7f7a69759700 1 -- 172.21.15.93:0/1696336221 >> 172.21.15.93:0/1696336221 conn(0x7f7a64100fd0 msgr2=0x7f7a64103400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.104+0000 7f7a69759700 1 -- 172.21.15.93:0/1696336221 shutdown_connections 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.104+0000 7f7a69759700 1 -- 172.21.15.93:0/1696336221 wait complete. 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.104+0000 7f7a69759700 1 Processor -- start 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.104+0000 7f7a69759700 1 -- start start 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.104+0000 7f7a69759700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a6407bc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.104+0000 7f7a69759700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a6407c140 con 0x7f7a64105a20 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a62ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a6407bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a62ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a6407bc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41354/0 (socket says 172.21.15.93:41354) 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a62ffd700 1 -- 172.21.15.93:0/3265319264 learned_addr learned my addr 172.21.15.93:0/3265319264 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a62ffd700 1 -- 172.21.15.93:0/3265319264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a58009550 con 0x7f7a64105a20 2023-12-14T18:19:40.663 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a62ffd700 1 --2- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a6407bc00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7a58000c00 tx=0x7f7a58011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a4bfff700 1 -- 172.21.15.93:0/3265319264 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a580039f0 con 0x7f7a64105a20 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a4bfff700 1 -- 172.21.15.93:0/3265319264 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a58003b50 con 0x7f7a64105a20 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a64109fe0 con 0x7f7a64105a20 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a4bfff700 1 -- 172.21.15.93:0/3265319264 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7a5801a780 con 0x7f7a64105a20 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.105+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a64109e50 con 0x7f7a64105a20 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.106+0000 7f7a4bfff700 1 -- 172.21.15.93:0/3265319264 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7f7a58024440 con 0x7f7a64105a20 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.106+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a6410a170 con 0x7f7a64105a20 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.106+0000 7f7a4bfff700 1 --2- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a4c038760 0x7f7a4c03ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.106+0000 7f7a4bfff700 1 -- 172.21.15.93:0/3265319264 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f7a5804c8d0 con 0x7f7a64105a20 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.106+0000 7f7a627fc700 1 --2- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a4c038760 0x7f7a4c03ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:40.664 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.107+0000 7f7a627fc700 1 --2- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a4c038760 0x7f7a4c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a54006fd0 tx=0x7f7a54006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.111+0000 7f7a4bfff700 1 -- 172.21.15.93:0/3265319264 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7a580189a0 con 0x7f7a64105a20 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.299+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7f7a6407a9b0 con 0x7f7a64105a20 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.300+0000 7f7a4bfff700 1 -- 172.21.15.93:0/3265319264 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7f7a58018350 con 0x7f7a64105a20 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a4c038760 msgr2=0x7f7a4c03ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 --2- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a4c038760 0x7f7a4c03ac10 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f7a54006fd0 tx=0x7f7a54006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 msgr2=0x7f7a6407bc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 --2- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a6407bc00 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f7a58000c00 tx=0x7f7a58011a30 comp rx=0 tx=0).stop 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 shutdown_connections 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 --2- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a4c038760 0x7f7a4c03ac10 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 --2- 172.21.15.93:0/3265319264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a64105a20 0x7f7a6407bc00 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:40.665 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 >> 172.21.15.93:0/3265319264 conn(0x7f7a64100fd0 msgr2=0x7f7a64102680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:40.666 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 shutdown_connections 2023-12-14T18:19:40.666 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:40.302+0000 7f7a69759700 1 -- 172.21.15.93:0/3265319264 wait complete. 2023-12-14T18:19:40.667 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2023-12-14T18:19:40.667 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout disabled 2023-12-14T18:19:40.674 INFO:teuthology.orchestra.run.smithi093.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2023-12-14T18:19:40.674 INFO:teuthology.orchestra.run.smithi093.stdout:systemctl: stdout inactive 2023-12-14T18:19:40.675 INFO:teuthology.orchestra.run.smithi093.stdout:firewalld.service is not enabled 2023-12-14T18:19:40.675 INFO:teuthology.orchestra.run.smithi093.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2023-12-14T18:19:40.676 INFO:teuthology.orchestra.run.smithi093.stdout:Ceph Dashboard is now available at: 2023-12-14T18:19:40.676 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:40.676 INFO:teuthology.orchestra.run.smithi093.stdout: URL: https://smithi093.front.sepia.ceph.com:8443/ 2023-12-14T18:19:40.677 INFO:teuthology.orchestra.run.smithi093.stdout: User: admin 2023-12-14T18:19:40.677 INFO:teuthology.orchestra.run.smithi093.stdout: Password: kqrtuoye6y 2023-12-14T18:19:40.677 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:40.677 INFO:teuthology.orchestra.run.smithi093.stdout:Saving cluster configuration to /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config directory 2023-12-14T18:19:40.677 INFO:teuthology.orchestra.run.smithi093.stdout:Enabling autotune for osd_memory_target 2023-12-14T18:19:41.428 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:41 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3265319264' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2023-12-14T18:19:41.626 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.055+0000 7efe0359e700 1 Processor -- start 2023-12-14T18:19:41.626 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.056+0000 7efe0359e700 1 -- start start 2023-12-14T18:19:41.626 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.056+0000 7efe0359e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe04079cb0 0x7efe04108460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:41.626 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.056+0000 7efe0359e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe041089a0 con 0x7efe04079cb0 2023-12-14T18:19:41.626 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.056+0000 7efe0259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe04079cb0 0x7efe04108460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:41.626 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.056+0000 7efe0259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe04079cb0 0x7efe04108460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41356/0 (socket says 172.21.15.93:41356) 2023-12-14T18:19:41.626 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.056+0000 7efe0259c700 1 -- 172.21.15.93:0/2315823611 learned_addr learned my addr 172.21.15.93:0/2315823611 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:41.626 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.056+0000 7efe0259c700 1 -- 172.21.15.93:0/2315823611 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe04108ae0 con 0x7efe04079cb0 2023-12-14T18:19:41.626 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0259c700 1 --2- 172.21.15.93:0/2315823611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe04079cb0 0x7efe04108460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efdf40098a0 tx=0x7efdf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6da8805aec656408 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0159a700 1 -- 172.21.15.93:0/2315823611 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdf4004160 con 0x7efe04079cb0 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0159a700 1 -- 172.21.15.93:0/2315823611 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdf40042c0 con 0x7efe04079cb0 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0159a700 1 -- 172.21.15.93:0/2315823611 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdf40119b0 con 0x7efe04079cb0 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0359e700 1 -- 172.21.15.93:0/2315823611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe04079cb0 msgr2=0x7efe04108460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0359e700 1 --2- 172.21.15.93:0/2315823611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe04079cb0 0x7efe04108460 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efdf40098a0 tx=0x7efdf4009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0359e700 1 -- 172.21.15.93:0/2315823611 shutdown_connections 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0359e700 1 --2- 172.21.15.93:0/2315823611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe04079cb0 0x7efe04108460 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0359e700 1 -- 172.21.15.93:0/2315823611 >> 172.21.15.93:0/2315823611 conn(0x7efe040ffe30 msgr2=0x7efe04102240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0359e700 1 -- 172.21.15.93:0/2315823611 shutdown_connections 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.057+0000 7efe0359e700 1 -- 172.21.15.93:0/2315823611 wait complete. 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.058+0000 7efe0359e700 1 Processor -- start 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.058+0000 7efe0359e700 1 -- start start 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.058+0000 7efe0359e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0419bb20 0x7efe0419bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:41.627 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.058+0000 7efe0359e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe0419c470 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.058+0000 7efe0259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0419bb20 0x7efe0419bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.058+0000 7efe0259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0419bb20 0x7efe0419bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:41358/0 (socket says 172.21.15.93:41358) 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.058+0000 7efe0259c700 1 -- 172.21.15.93:0/1008682370 learned_addr learned my addr 172.21.15.93:0/1008682370 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.059+0000 7efe0259c700 1 -- 172.21.15.93:0/1008682370 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf4009550 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.059+0000 7efe0259c700 1 --2- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0419bb20 0x7efe0419bf30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efdf4003e60 tx=0x7efdf4011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.059+0000 7efdf37fe700 1 -- 172.21.15.93:0/1008682370 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdf4003a60 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.059+0000 7efdf37fe700 1 -- 172.21.15.93:0/1008682370 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efdf4003bc0 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.059+0000 7efdf37fe700 1 -- 172.21.15.93:0/1008682370 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7efdf40196c0 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.059+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe04109b70 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.059+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe041099e0 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.060+0000 7efdf37fe700 1 -- 172.21.15.93:0/1008682370 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7efdf401ab60 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.060+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe04195a60 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.060+0000 7efdf37fe700 1 --2- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efdec038740 0x7efdec03abf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.060+0000 7efdf37fe700 1 -- 172.21.15.93:0/1008682370 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7efdf404cab0 con 0x7efe0419bb20 2023-12-14T18:19:41.628 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.060+0000 7efe01d9b700 1 --2- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efdec038740 0x7efdec03abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.061+0000 7efe01d9b700 1 --2- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efdec038740 0x7efdec03abf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efdf8006fd0 tx=0x7efdf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.069+0000 7efdf37fe700 1 -- 172.21.15.93:0/1008682370 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efdf4010790 con 0x7efe0419bb20 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.263+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7efe0402cfe0 con 0x7efe0419bb20 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.263+0000 7efdf37fe700 1 -- 172.21.15.93:0/1008682370 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7efdf4010380 con 0x7efe0419bb20 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efdec038740 msgr2=0x7efdec03abf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 --2- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efdec038740 0x7efdec03abf0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7efdf8006fd0 tx=0x7efdf8006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0419bb20 msgr2=0x7efe0419bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 --2- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0419bb20 0x7efe0419bf30 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efdf4003e60 tx=0x7efdf4011a30 comp rx=0 tx=0).stop 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 shutdown_connections 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 --2- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efdec038740 0x7efdec03abf0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 --2- 172.21.15.93:0/1008682370 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efe0419bb20 0x7efe0419bf30 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 >> 172.21.15.93:0/1008682370 conn(0x7efe040ffe30 msgr2=0x7efe04102160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 shutdown_connections 2023-12-14T18:19:41.629 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.265+0000 7efe0359e700 1 -- 172.21.15.93:0/1008682370 wait complete. 2023-12-14T18:19:42.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.985+0000 7fbf07de1700 1 Processor -- start 2023-12-14T18:19:42.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.985+0000 7fbf07de1700 1 -- start start 2023-12-14T18:19:42.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.986+0000 7fbf07de1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00108870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:42.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.986+0000 7fbf07de1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf00079e60 con 0x7fbf00106490 2023-12-14T18:19:42.645 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.986+0000 7fbf05b7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00108870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.986+0000 7fbf05b7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00108870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38454/0 (socket says 172.21.15.93:38454) 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.986+0000 7fbf05b7d700 1 -- 172.21.15.93:0/2064470031 learned_addr learned my addr 172.21.15.93:0/2064470031 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.986+0000 7fbf05b7d700 1 -- 172.21.15.93:0/2064470031 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf00108db0 con 0x7fbf00106490 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.986+0000 7fbf05b7d700 1 --2- 172.21.15.93:0/2064470031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00108870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbef00098a0 tx=0x7fbef0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba828a59b68ad885 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.986+0000 7fbf04b7b700 1 -- 172.21.15.93:0/2064470031 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbef0004160 con 0x7fbf00106490 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.986+0000 7fbf04b7b700 1 -- 172.21.15.93:0/2064470031 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbef00042c0 con 0x7fbf00106490 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.987+0000 7fbf04b7b700 1 -- 172.21.15.93:0/2064470031 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbef00119b0 con 0x7fbf00106490 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.987+0000 7fbf07de1700 1 -- 172.21.15.93:0/2064470031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 msgr2=0x7fbf00108870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.987+0000 7fbf07de1700 1 --2- 172.21.15.93:0/2064470031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00108870 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fbef00098a0 tx=0x7fbef0009bb0 comp rx=0 tx=0).stop 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.987+0000 7fbf07de1700 1 -- 172.21.15.93:0/2064470031 shutdown_connections 2023-12-14T18:19:42.646 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.987+0000 7fbf07de1700 1 --2- 172.21.15.93:0/2064470031 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00108870 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.987+0000 7fbf07de1700 1 -- 172.21.15.93:0/2064470031 >> 172.21.15.93:0/2064470031 conn(0x7fbf000ffed0 msgr2=0x7fbf00102300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.987+0000 7fbf07de1700 1 -- 172.21.15.93:0/2064470031 shutdown_connections 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.987+0000 7fbf07de1700 1 -- 172.21.15.93:0/2064470031 wait complete. 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbf07de1700 1 Processor -- start 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbf07de1700 1 -- start start 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbf07de1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00199780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbf07de1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf00199cc0 con 0x7fbf00106490 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbf05b7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00199780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbf05b7d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00199780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38464/0 (socket says 172.21.15.93:38464) 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbf05b7d700 1 -- 172.21.15.93:0/3439550274 learned_addr learned my addr 172.21.15.93:0/3439550274 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbf05b7d700 1 -- 172.21.15.93:0/3439550274 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbef0009550 con 0x7fbf00106490 2023-12-14T18:19:42.647 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbf05b7d700 1 --2- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00199780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbef0011980 tx=0x7fbef0011a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.988+0000 7fbef6ffd700 1 -- 172.21.15.93:0/3439550274 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbef0003a60 con 0x7fbf00106490 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.989+0000 7fbef6ffd700 1 -- 172.21.15.93:0/3439550274 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbef0003bc0 con 0x7fbf00106490 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.989+0000 7fbef6ffd700 1 -- 172.21.15.93:0/3439550274 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fbef001a670 con 0x7fbf00106490 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.989+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf00109d80 con 0x7fbf00106490 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.989+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf00109bf0 con 0x7fbf00106490 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.989+0000 7fbef6ffd700 1 -- 172.21.15.93:0/3439550274 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 12) v1 ==== 45496+0+0 (secure 0 0 0) 0x7fbef001a8e0 con 0x7fbf00106490 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.989+0000 7fbef6ffd700 1 --2- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbeec0387b0 0x7fbeec03ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.989+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf00109f10 con 0x7fbf00106490 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.990+0000 7fbef6ffd700 1 -- 172.21.15.93:0/3439550274 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fbef004c8a0 con 0x7fbf00106490 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.990+0000 7fbf0537c700 1 --2- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbeec0387b0 0x7fbeec03ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.990+0000 7fbf0537c700 1 --2- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbeec0387b0 0x7fbeec03ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbefc006fd0 tx=0x7fbefc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:42.648 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:41.994+0000 7fbef6ffd700 1 -- 172.21.15.93:0/3439550274 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbef0024a00 con 0x7fbf00106490 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.119+0000 7fbef6ffd700 1 -- 172.21.15.93:0/3439550274 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fbef001fd90 con 0x7fbf00106490 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.285+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7fbf0019d390 con 0x7fbf00106490 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.287+0000 7fbef6ffd700 1 -- 172.21.15.93:0/3439550274 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7fbef002ad60 con 0x7fbf00106490 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbeec0387b0 msgr2=0x7fbeec03ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 --2- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbeec0387b0 0x7fbeec03ac60 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fbefc006fd0 tx=0x7fbefc006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 msgr2=0x7fbf00199780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 --2- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00199780 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbef0011980 tx=0x7fbef0011a60 comp rx=0 tx=0).stop 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 shutdown_connections 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 --2- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbeec0387b0 0x7fbeec03ac60 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 --2- 172.21.15.93:0/3439550274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbf00106490 0x7fbf00199780 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:42.649 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 >> 172.21.15.93:0/3439550274 conn(0x7fbf000ffed0 msgr2=0x7fbf00100cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 shutdown_connections 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr 2023-12-14T18:19:42.289+0000 7fbf07de1700 1 -- 172.21.15.93:0/3439550274 wait complete. 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout:Or, if you are only running a single cluster on this host: 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout:Please consider enabling telemetry to help improve Ceph: 2023-12-14T18:19:42.650 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:42.651 INFO:teuthology.orchestra.run.smithi093.stdout: ceph telemetry on 2023-12-14T18:19:42.651 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:42.651 INFO:teuthology.orchestra.run.smithi093.stdout:For more information see: 2023-12-14T18:19:42.651 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:42.651 INFO:teuthology.orchestra.run.smithi093.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2023-12-14T18:19:42.651 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:42.651 INFO:teuthology.orchestra.run.smithi093.stdout:Bootstrap complete. 2023-12-14T18:19:42.675 INFO:tasks.cephadm:Fetching config... 2023-12-14T18:19:42.675 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:19:42.675 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2023-12-14T18:19:42.691 INFO:tasks.cephadm:Fetching client.admin keyring... 2023-12-14T18:19:42.692 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:19:42.692 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2023-12-14T18:19:42.748 INFO:tasks.cephadm:Fetching mon keyring... 2023-12-14T18:19:42.749 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:19:42.749 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/keyring of=/dev/stdout 2023-12-14T18:19:42.816 INFO:tasks.cephadm:Fetching pub ssh key... 2023-12-14T18:19:42.816 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:19:42.816 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2023-12-14T18:19:42.875 INFO:tasks.cephadm:Installing pub ssh key for root users... 2023-12-14T18:19:42.875 DEBUG:teuthology.orchestra.run.smithi093:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwpfVZRML/QmaCwEm4CeUHs1hwpffM4F7L7DbjZ3c1z3igmNNVbxQefHJGjjMdKb0RoEiOp/e6WuisxpqxqNgwEfGWfAMP0X3qT+V6yN8kddgyxfX/hmJbsRnRMUBQ07Mz148we93kWiqOrf4W2JQfr3sPbGjB+ibgil+0/3ZZSovq98lOwA4sMmDOuy3p5Y0Nl7UfUe0g4Pj7a051yxcHzdKRKpUo//yyXYnjpmkZvw1K0w6aED5aFniZh3NG4hfEvFJe78/mUrs9J+5sNZowOppBGRf6NDVaLbt1a6mlx5ZxOVU+KoSkC04o2fhABIDYoT464g9oNVuickr0dT1p5MmO6odd3lFVIsxJmda5quECJlHf2HUUfmZ1hTm8mReq2NCo931X5M5PG8GK7GL1E4wbmVsU2IUnQRPo6RT1bo6fdeo6QyxwQh0G3n3a1+uX3qGvJtZNH5VE8g4ezAQzo4DgsIFs070F/w0YO37JYw0oN+V2E9DRYQlPbl8hgdk= ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-14T18:19:42.953 INFO:teuthology.orchestra.run.smithi093.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwpfVZRML/QmaCwEm4CeUHs1hwpffM4F7L7DbjZ3c1z3igmNNVbxQefHJGjjMdKb0RoEiOp/e6WuisxpqxqNgwEfGWfAMP0X3qT+V6yN8kddgyxfX/hmJbsRnRMUBQ07Mz148we93kWiqOrf4W2JQfr3sPbGjB+ibgil+0/3ZZSovq98lOwA4sMmDOuy3p5Y0Nl7UfUe0g4Pj7a051yxcHzdKRKpUo//yyXYnjpmkZvw1K0w6aED5aFniZh3NG4hfEvFJe78/mUrs9J+5sNZowOppBGRf6NDVaLbt1a6mlx5ZxOVU+KoSkC04o2fhABIDYoT464g9oNVuickr0dT1p5MmO6odd3lFVIsxJmda5quECJlHf2HUUfmZ1hTm8mReq2NCo931X5M5PG8GK7GL1E4wbmVsU2IUnQRPo6RT1bo6fdeo6QyxwQh0G3n3a1+uX3qGvJtZNH5VE8g4ezAQzo4DgsIFs070F/w0YO37JYw0oN+V2E9DRYQlPbl8hgdk= ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:42.965 DEBUG:teuthology.orchestra.run.smithi097:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwpfVZRML/QmaCwEm4CeUHs1hwpffM4F7L7DbjZ3c1z3igmNNVbxQefHJGjjMdKb0RoEiOp/e6WuisxpqxqNgwEfGWfAMP0X3qT+V6yN8kddgyxfX/hmJbsRnRMUBQ07Mz148we93kWiqOrf4W2JQfr3sPbGjB+ibgil+0/3ZZSovq98lOwA4sMmDOuy3p5Y0Nl7UfUe0g4Pj7a051yxcHzdKRKpUo//yyXYnjpmkZvw1K0w6aED5aFniZh3NG4hfEvFJe78/mUrs9J+5sNZowOppBGRf6NDVaLbt1a6mlx5ZxOVU+KoSkC04o2fhABIDYoT464g9oNVuickr0dT1p5MmO6odd3lFVIsxJmda5quECJlHf2HUUfmZ1hTm8mReq2NCo931X5M5PG8GK7GL1E4wbmVsU2IUnQRPo6RT1bo6fdeo6QyxwQh0G3n3a1+uX3qGvJtZNH5VE8g4ezAQzo4DgsIFs070F/w0YO37JYw0oN+V2E9DRYQlPbl8hgdk= ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-14T18:19:43.010 INFO:teuthology.orchestra.run.smithi097.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwpfVZRML/QmaCwEm4CeUHs1hwpffM4F7L7DbjZ3c1z3igmNNVbxQefHJGjjMdKb0RoEiOp/e6WuisxpqxqNgwEfGWfAMP0X3qT+V6yN8kddgyxfX/hmJbsRnRMUBQ07Mz148we93kWiqOrf4W2JQfr3sPbGjB+ibgil+0/3ZZSovq98lOwA4sMmDOuy3p5Y0Nl7UfUe0g4Pj7a051yxcHzdKRKpUo//yyXYnjpmkZvw1K0w6aED5aFniZh3NG4hfEvFJe78/mUrs9J+5sNZowOppBGRf6NDVaLbt1a6mlx5ZxOVU+KoSkC04o2fhABIDYoT464g9oNVuickr0dT1p5MmO6odd3lFVIsxJmda5quECJlHf2HUUfmZ1hTm8mReq2NCo931X5M5PG8GK7GL1E4wbmVsU2IUnQRPo6RT1bo6fdeo6QyxwQh0G3n3a1+uX3qGvJtZNH5VE8g4ezAQzo4DgsIFs070F/w0YO37JYw0oN+V2E9DRYQlPbl8hgdk= ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:43.023 DEBUG:teuthology.orchestra.run.smithi160:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwpfVZRML/QmaCwEm4CeUHs1hwpffM4F7L7DbjZ3c1z3igmNNVbxQefHJGjjMdKb0RoEiOp/e6WuisxpqxqNgwEfGWfAMP0X3qT+V6yN8kddgyxfX/hmJbsRnRMUBQ07Mz148we93kWiqOrf4W2JQfr3sPbGjB+ibgil+0/3ZZSovq98lOwA4sMmDOuy3p5Y0Nl7UfUe0g4Pj7a051yxcHzdKRKpUo//yyXYnjpmkZvw1K0w6aED5aFniZh3NG4hfEvFJe78/mUrs9J+5sNZowOppBGRf6NDVaLbt1a6mlx5ZxOVU+KoSkC04o2fhABIDYoT464g9oNVuickr0dT1p5MmO6odd3lFVIsxJmda5quECJlHf2HUUfmZ1hTm8mReq2NCo931X5M5PG8GK7GL1E4wbmVsU2IUnQRPo6RT1bo6fdeo6QyxwQh0G3n3a1+uX3qGvJtZNH5VE8g4ezAQzo4DgsIFs070F/w0YO37JYw0oN+V2E9DRYQlPbl8hgdk= ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2023-12-14T18:19:43.068 INFO:teuthology.orchestra.run.smithi160.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCwpfVZRML/QmaCwEm4CeUHs1hwpffM4F7L7DbjZ3c1z3igmNNVbxQefHJGjjMdKb0RoEiOp/e6WuisxpqxqNgwEfGWfAMP0X3qT+V6yN8kddgyxfX/hmJbsRnRMUBQ07Mz148we93kWiqOrf4W2JQfr3sPbGjB+ibgil+0/3ZZSovq98lOwA4sMmDOuy3p5Y0Nl7UfUe0g4Pj7a051yxcHzdKRKpUo//yyXYnjpmkZvw1K0w6aED5aFniZh3NG4hfEvFJe78/mUrs9J+5sNZowOppBGRf6NDVaLbt1a6mlx5ZxOVU+KoSkC04o2fhABIDYoT464g9oNVuickr0dT1p5MmO6odd3lFVIsxJmda5quECJlHf2HUUfmZ1hTm8mReq2NCo931X5M5PG8GK7GL1E4wbmVsU2IUnQRPo6RT1bo6fdeo6QyxwQh0G3n3a1+uX3qGvJtZNH5VE8g4ezAQzo4DgsIFs070F/w0YO37JYw0oN+V2E9DRYQlPbl8hgdk= ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:19:43.081 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2023-12-14T18:19:43.402 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:19:43.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:43 smithi093 ceph-mon[42070]: mgrmap e13: x(active, since 6s) 2023-12-14T18:19:43.420 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:43 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3439550274' entity='client.admin' 2023-12-14T18:19:44.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.639+0000 7fd8a93db700 1 -- 172.21.15.93:0/2943265619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 msgr2=0x7fd8a4100130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:44.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.639+0000 7fd8a93db700 1 --2- 172.21.15.93:0/2943265619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 0x7fd8a4100130 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fd894009910 tx=0x7fd894009c20 comp rx=0 tx=0).stop 2023-12-14T18:19:44.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.639+0000 7fd8a93db700 1 -- 172.21.15.93:0/2943265619 shutdown_connections 2023-12-14T18:19:44.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.639+0000 7fd8a93db700 1 --2- 172.21.15.93:0/2943265619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 0x7fd8a4100130 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:44.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.639+0000 7fd8a93db700 1 -- 172.21.15.93:0/2943265619 >> 172.21.15.93:0/2943265619 conn(0x7fd8a40fb2d0 msgr2=0x7fd8a40fd700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:44.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.639+0000 7fd8a93db700 1 -- 172.21.15.93:0/2943265619 shutdown_connections 2023-12-14T18:19:44.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.639+0000 7fd8a93db700 1 -- 172.21.15.93:0/2943265619 wait complete. 2023-12-14T18:19:44.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.639+0000 7fd8a93db700 1 Processor -- start 2023-12-14T18:19:44.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.640+0000 7fd8a93db700 1 -- start start 2023-12-14T18:19:44.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.640+0000 7fd8a93db700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 0x7fd8a4074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:44.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.640+0000 7fd8a93db700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8a4074dd0 con 0x7fd8a40ffd20 2023-12-14T18:19:44.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.640+0000 7fd8a2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 0x7fd8a4074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:44.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.640+0000 7fd8a2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 0x7fd8a4074890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38482/0 (socket says 172.21.15.93:38482) 2023-12-14T18:19:44.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.640+0000 7fd8a2ffd700 1 -- 172.21.15.93:0/2591038864 learned_addr learned my addr 172.21.15.93:0/2591038864 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:44.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.640+0000 7fd8a2ffd700 1 -- 172.21.15.93:0/2591038864 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8940095f0 con 0x7fd8a40ffd20 2023-12-14T18:19:44.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.641+0000 7fd8a2ffd700 1 --2- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 0x7fd8a4074890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd894006010 tx=0x7fd8940057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:44.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.641+0000 7fd88bfff700 1 -- 172.21.15.93:0/2591038864 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd89401d070 con 0x7fd8a40ffd20 2023-12-14T18:19:44.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.641+0000 7fd88bfff700 1 -- 172.21.15.93:0/2591038864 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd8940044e0 con 0x7fd8a40ffd20 2023-12-14T18:19:44.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.641+0000 7fd88bfff700 1 -- 172.21.15.93:0/2591038864 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd8940039b0 con 0x7fd8a40ffd20 2023-12-14T18:19:44.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.641+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a4105bf0 con 0x7fd8a40ffd20 2023-12-14T18:19:44.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.641+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8a4105a60 con 0x7fd8a40ffd20 2023-12-14T18:19:44.644 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.642+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8a4062120 con 0x7fd8a40ffd20 2023-12-14T18:19:44.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.642+0000 7fd88bfff700 1 -- 172.21.15.93:0/2591038864 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fd89400baf0 con 0x7fd8a40ffd20 2023-12-14T18:19:44.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.642+0000 7fd88bfff700 1 --2- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd88c038870 0x7fd88c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:44.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.642+0000 7fd88bfff700 1 -- 172.21.15.93:0/2591038864 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fd89404cdc0 con 0x7fd8a40ffd20 2023-12-14T18:19:44.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.643+0000 7fd8a27fc700 1 --2- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd88c038870 0x7fd88c03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:44.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.643+0000 7fd8a27fc700 1 --2- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd88c038870 0x7fd88c03ad20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd898006fd0 tx=0x7fd898006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:44.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.647+0000 7fd88bfff700 1 -- 172.21.15.93:0/2591038864 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd89400e9b0 con 0x7fd8a40ffd20 2023-12-14T18:19:44.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.844+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fd8a418e300 con 0x7fd8a40ffd20 2023-12-14T18:19:44.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.847+0000 7fd88bfff700 1 -- 172.21.15.93:0/2591038864 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fd89400e3e0 con 0x7fd8a40ffd20 2023-12-14T18:19:44.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.849+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd88c038870 msgr2=0x7fd88c03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:44.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.849+0000 7fd8a93db700 1 --2- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd88c038870 0x7fd88c03ad20 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd898006fd0 tx=0x7fd898006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:44.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.849+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 msgr2=0x7fd8a4074890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:44.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.849+0000 7fd8a93db700 1 --2- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 0x7fd8a4074890 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd894006010 tx=0x7fd8940057c0 comp rx=0 tx=0).stop 2023-12-14T18:19:44.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.849+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 shutdown_connections 2023-12-14T18:19:44.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.849+0000 7fd8a93db700 1 --2- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd88c038870 0x7fd88c03ad20 secure :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fd898006fd0 tx=0x7fd898006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:44.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.849+0000 7fd8a93db700 1 --2- 172.21.15.93:0/2591038864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd8a40ffd20 0x7fd8a4074890 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:44.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.849+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 >> 172.21.15.93:0/2591038864 conn(0x7fd8a40fb2d0 msgr2=0x7fd8a40fbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:44.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.850+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 shutdown_connections 2023-12-14T18:19:44.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:44.850+0000 7fd8a93db700 1 -- 172.21.15.93:0/2591038864 wait complete. 2023-12-14T18:19:45.206 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:45.206 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:45.206 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi093", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:19:45.206 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:45.206 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:45.206 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:45.206 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:45 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2591038864' entity='client.admin' 2023-12-14T18:19:45.206 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:45.206 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:45.421 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2023-12-14T18:19:45.421 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch client-keyring set client.admin '*' --mode 0755 2023-12-14T18:19:45.727 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:19:46.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.631+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/1217111574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 msgr2=0x7fa0e41017a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:46.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.631+0000 7fa0ea7dc700 1 --2- 172.21.15.93:0/1217111574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 0x7fa0e41017a0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009910 tx=0x7fa0d4009c20 comp rx=0 tx=0).stop 2023-12-14T18:19:46.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.631+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/1217111574 shutdown_connections 2023-12-14T18:19:46.633 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.631+0000 7fa0ea7dc700 1 --2- 172.21.15.93:0/1217111574 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 0x7fa0e41017a0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:46.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.631+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/1217111574 >> 172.21.15.93:0/1217111574 conn(0x7fa0e40f99e0 msgr2=0x7fa0e40fbdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:46.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.631+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/1217111574 shutdown_connections 2023-12-14T18:19:46.634 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.631+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/1217111574 wait complete. 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.632+0000 7fa0ea7dc700 1 Processor -- start 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.632+0000 7fa0ea7dc700 1 -- start start 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.632+0000 7fa0ea7dc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 0x7fa0e4194e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0ea7dc700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0e4195340 con 0x7fa0e4101390 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0e3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 0x7fa0e4194e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0e3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 0x7fa0e4194e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38506/0 (socket says 172.21.15.93:38506) 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0e3fff700 1 -- 172.21.15.93:0/3239185708 learned_addr learned my addr 172.21.15.93:0/3239185708 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0e3fff700 1 -- 172.21.15.93:0/3239185708 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40095f0 con 0x7fa0e4101390 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0e3fff700 1 --2- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 0x7fa0e4194e00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4004f40 tx=0x7fa0d40057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:46.635 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0e17fa700 1 -- 172.21.15.93:0/3239185708 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0d401d070 con 0x7fa0e4101390 2023-12-14T18:19:46.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0e17fa700 1 -- 172.21.15.93:0/3239185708 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0d40044e0 con 0x7fa0e4101390 2023-12-14T18:19:46.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0e4105b40 con 0x7fa0e4101390 2023-12-14T18:19:46.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0e17fa700 1 -- 172.21.15.93:0/3239185708 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0d4003a20 con 0x7fa0e4101390 2023-12-14T18:19:46.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.633+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0e41059b0 con 0x7fa0e4101390 2023-12-14T18:19:46.636 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.634+0000 7fa0e17fa700 1 -- 172.21.15.93:0/3239185708 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fa0d4003ca0 con 0x7fa0e4101390 2023-12-14T18:19:46.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.634+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0e4105cd0 con 0x7fa0e4101390 2023-12-14T18:19:46.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.634+0000 7fa0e17fa700 1 --2- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0d0038870 0x7fa0d003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:46.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.634+0000 7fa0e17fa700 1 -- 172.21.15.93:0/3239185708 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa0d404ca40 con 0x7fa0e4101390 2023-12-14T18:19:46.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.634+0000 7fa0e37fe700 1 --2- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0d0038870 0x7fa0d003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:46.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.635+0000 7fa0e37fe700 1 --2- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0d0038870 0x7fa0d003ad20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc006fd0 tx=0x7fa0cc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:46.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.639+0000 7fa0e17fa700 1 -- 172.21.15.93:0/3239185708 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa0d404ee60 con 0x7fa0e4101390 2023-12-14T18:19:46.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.829+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7fa0e4060f30 con 0x7fa0d0038870 2023-12-14T18:19:46.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.832+0000 7fa0e17fa700 1 -- 172.21.15.93:0/3239185708 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7fa0e4060f30 con 0x7fa0d0038870 2023-12-14T18:19:46.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.834+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0d0038870 msgr2=0x7fa0d003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:46.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.834+0000 7fa0ea7dc700 1 --2- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0d0038870 0x7fa0d003ad20 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fa0cc006fd0 tx=0x7fa0cc006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:46.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.834+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 msgr2=0x7fa0e4194e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:46.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.834+0000 7fa0ea7dc700 1 --2- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 0x7fa0e4194e00 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4004f40 tx=0x7fa0d40057c0 comp rx=0 tx=0).stop 2023-12-14T18:19:46.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.835+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 shutdown_connections 2023-12-14T18:19:46.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.835+0000 7fa0ea7dc700 1 --2- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0d0038870 0x7fa0d003ad20 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:46.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.835+0000 7fa0ea7dc700 1 --2- 172.21.15.93:0/3239185708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0e4101390 0x7fa0e4194e00 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:46.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.835+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 >> 172.21.15.93:0/3239185708 conn(0x7fa0e40f99e0 msgr2=0x7fa0e40fa660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:46.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.835+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 shutdown_connections 2023-12-14T18:19:46.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:46.835+0000 7fa0ea7dc700 1 -- 172.21.15.93:0/3239185708 wait complete. 2023-12-14T18:19:47.202 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi097 2023-12-14T18:19:47.202 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:19:47.202 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.conf 2023-12-14T18:19:47.223 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:19:47.223 DEBUG:teuthology.orchestra.run.smithi097:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:19:47.283 INFO:tasks.cephadm:Adding host smithi097 to orchestrator... 2023-12-14T18:19:47.284 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch host add smithi097 2023-12-14T18:19:47.610 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:19:48.002 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:48.002 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:48.002 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:48.002 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:19:48.002 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:19:48.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:19:48.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: Updating smithi093:/etc/ceph/ceph.conf 2023-12-14T18:19:48.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:19:48.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:19:48.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:48.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:48.003 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:47 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:48.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.512+0000 7f5857516700 1 -- 172.21.15.93:0/492205051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 msgr2=0x7f58500ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:48.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.512+0000 7f5857516700 1 --2- 172.21.15.93:0/492205051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 0x7f58500ff140 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5844009910 tx=0x7f5844009c20 comp rx=0 tx=0).stop 2023-12-14T18:19:48.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.512+0000 7f5857516700 1 -- 172.21.15.93:0/492205051 shutdown_connections 2023-12-14T18:19:48.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.512+0000 7f5857516700 1 --2- 172.21.15.93:0/492205051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 0x7f58500ff140 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:48.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.512+0000 7f5857516700 1 -- 172.21.15.93:0/492205051 >> 172.21.15.93:0/492205051 conn(0x7f58500763d0 msgr2=0x7f58500767d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.512+0000 7f5857516700 1 -- 172.21.15.93:0/492205051 shutdown_connections 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.512+0000 7f5857516700 1 -- 172.21.15.93:0/492205051 wait complete. 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f5857516700 1 Processor -- start 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f5857516700 1 -- start start 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f5857516700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 0x7f5850197380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f5857516700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58501978c0 con 0x7f58500fed70 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f58552b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 0x7f5850197380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f58552b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 0x7f5850197380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38512/0 (socket says 172.21.15.93:38512) 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f58552b2700 1 -- 172.21.15.93:0/1565126083 learned_addr learned my addr 172.21.15.93:0/1565126083 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f58552b2700 1 -- 172.21.15.93:0/1565126083 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58440095f0 con 0x7f58500fed70 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f58552b2700 1 --2- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 0x7f5850197380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5844004f40 tx=0x7f5844005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f58427fc700 1 -- 172.21.15.93:0/1565126083 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f584401d070 con 0x7f58500fed70 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f58427fc700 1 -- 172.21.15.93:0/1565126083 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5844004370 con 0x7f58500fed70 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f585010a260 con 0x7f58500fed70 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.513+0000 7f58427fc700 1 -- 172.21.15.93:0/1565126083 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f58440039b0 con 0x7f58500fed70 2023-12-14T18:19:48.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.514+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f585010a0d0 con 0x7f58500fed70 2023-12-14T18:19:48.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.514+0000 7f58427fc700 1 -- 172.21.15.93:0/1565126083 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f5844003c70 con 0x7f58500fed70 2023-12-14T18:19:48.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.514+0000 7f58427fc700 1 --2- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f583c038870 0x7f583c03ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:48.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.514+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f585010a3f0 con 0x7f58500fed70 2023-12-14T18:19:48.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.515+0000 7f58427fc700 1 -- 172.21.15.93:0/1565126083 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f584404c7c0 con 0x7f58500fed70 2023-12-14T18:19:48.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.515+0000 7f5854ab1700 1 --2- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f583c038870 0x7f583c03ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:48.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.515+0000 7f5854ab1700 1 --2- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f583c038870 0x7f583c03ad20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f584c006fd0 tx=0x7f584c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:48.521 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.519+0000 7f58427fc700 1 -- 172.21.15.93:0/1565126083 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5844026070 con 0x7f58500fed70 2023-12-14T18:19:48.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:48.710+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}) v1 -- 0x7f5850197e70 con 0x7f583c038870 2023-12-14T18:19:49.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:48 smithi093 ceph-mon[42070]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:19:50.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:49 smithi093 ceph-mon[42070]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi097", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:50.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:49 smithi093 ceph-mon[42070]: Deploying cephadm binary to smithi097 2023-12-14T18:19:50.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.605+0000 7f58427fc700 1 -- 172.21.15.93:0/1565126083 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+48 (secure 0 0 0) 0x7f5850197e70 con 0x7f583c038870 2023-12-14T18:19:50.607 INFO:teuthology.orchestra.run.smithi093.stdout:Added host 'smithi097' with addr '172.21.15.97' 2023-12-14T18:19:50.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f583c038870 msgr2=0x7f583c03ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:50.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 --2- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f583c038870 0x7f583c03ad20 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f584c006fd0 tx=0x7f584c006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:50.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 msgr2=0x7f5850197380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:50.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 --2- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 0x7f5850197380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f5844004f40 tx=0x7f5844005020 comp rx=0 tx=0).stop 2023-12-14T18:19:50.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 shutdown_connections 2023-12-14T18:19:50.609 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 --2- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f583c038870 0x7f583c03ad20 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:50.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 --2- 172.21.15.93:0/1565126083 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58500fed70 0x7f5850197380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:50.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 >> 172.21.15.93:0/1565126083 conn(0x7f58500763d0 msgr2=0x7f5850106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:50.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 shutdown_connections 2023-12-14T18:19:50.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:50.608+0000 7f5857516700 1 -- 172.21.15.93:0/1565126083 wait complete. 2023-12-14T18:19:51.024 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch host ls --format=json 2023-12-14T18:19:51.328 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:19:51.776 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:51 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:51.776 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:51 smithi093 ceph-mon[42070]: Added host smithi097 2023-12-14T18:19:51.776 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:51 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:51.776 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:51 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:51.776 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:51 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.353+0000 7fa0fd12c700 1 -- 172.21.15.93:0/735733179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 msgr2=0x7fa0f80ff350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.353+0000 7fa0fd12c700 1 --2- 172.21.15.93:0/735733179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 0x7fa0f80ff350 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec009910 tx=0x7fa0ec009c20 comp rx=0 tx=0).stop 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.353+0000 7fa0fd12c700 1 -- 172.21.15.93:0/735733179 shutdown_connections 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.353+0000 7fa0fd12c700 1 --2- 172.21.15.93:0/735733179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 0x7fa0f80ff350 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.353+0000 7fa0fd12c700 1 -- 172.21.15.93:0/735733179 >> 172.21.15.93:0/735733179 conn(0x7fa0f80fa880 msgr2=0x7fa0f80fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.353+0000 7fa0fd12c700 1 -- 172.21.15.93:0/735733179 shutdown_connections 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.353+0000 7fa0fd12c700 1 -- 172.21.15.93:0/735733179 wait complete. 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.353+0000 7fa0fd12c700 1 Processor -- start 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0fd12c700 1 -- start start 2023-12-14T18:19:52.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0fd12c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 0x7fa0f81951a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0fd12c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0f81956e0 con 0x7fa0f80fef80 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0f6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 0x7fa0f81951a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0f6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 0x7fa0f81951a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39600/0 (socket says 172.21.15.93:39600) 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0f6d9d700 1 -- 172.21.15.93:0/979708918 learned_addr learned my addr 172.21.15.93:0/979708918 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0f6d9d700 1 -- 172.21.15.93:0/979708918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0ec0095f0 con 0x7fa0f80fef80 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0f6d9d700 1 --2- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 0x7fa0f81951a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec004f40 tx=0x7fa0ec005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0dffff700 1 -- 172.21.15.93:0/979708918 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ec01d070 con 0x7fa0f80fef80 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0f8109160 con 0x7fa0f80fef80 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0dffff700 1 -- 172.21.15.93:0/979708918 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0ec004370 con 0x7fa0f80fef80 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.354+0000 7fa0dffff700 1 -- 172.21.15.93:0/979708918 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa0ec0039b0 con 0x7fa0f80fef80 2023-12-14T18:19:52.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.355+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0f8108fd0 con 0x7fa0f80fef80 2023-12-14T18:19:52.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.355+0000 7fa0dffff700 1 -- 172.21.15.93:0/979708918 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fa0ec003c70 con 0x7fa0f80fef80 2023-12-14T18:19:52.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.355+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0f81092f0 con 0x7fa0f80fef80 2023-12-14T18:19:52.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.355+0000 7fa0dffff700 1 --2- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0e0038870 0x7fa0e003ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:52.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.355+0000 7fa0dffff700 1 -- 172.21.15.93:0/979708918 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fa0ec04c630 con 0x7fa0f80fef80 2023-12-14T18:19:52.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.356+0000 7fa0f659c700 1 --2- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0e0038870 0x7fa0e003ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:52.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.356+0000 7fa0f659c700 1 --2- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0e0038870 0x7fa0e003ad20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8006fd0 tx=0x7fa0e8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:52.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.360+0000 7fa0dffff700 1 -- 172.21.15.93:0/979708918 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa0ec029770 con 0x7fa0f80fef80 2023-12-14T18:19:52.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.552+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa0f8195cc0 con 0x7fa0e0038870 2023-12-14T18:19:52.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.553+0000 7fa0dffff700 1 -- 172.21.15.93:0/979708918 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+159 (secure 0 0 0) 0x7fa0f8195cc0 con 0x7fa0e0038870 2023-12-14T18:19:52.553 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:52.554 INFO:teuthology.orchestra.run.smithi093.stdout:[{"addr": "172.21.15.93", "hostname": "smithi093", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}] 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0e0038870 msgr2=0x7fa0e003ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 --2- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0e0038870 0x7fa0e003ad20 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8006fd0 tx=0x7fa0e8006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 msgr2=0x7fa0f81951a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 --2- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 0x7fa0f81951a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa0ec004f40 tx=0x7fa0ec005020 comp rx=0 tx=0).stop 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 shutdown_connections 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 --2- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0e0038870 0x7fa0e003ad20 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 --2- 172.21.15.93:0/979708918 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0f80fef80 0x7fa0f81951a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 >> 172.21.15.93:0/979708918 conn(0x7fa0f80fa880 msgr2=0x7fa0f8068580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 shutdown_connections 2023-12-14T18:19:52.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:52.555+0000 7fa0fd12c700 1 -- 172.21.15.93:0/979708918 wait complete. 2023-12-14T18:19:52.954 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi160 2023-12-14T18:19:52.954 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:19:52.955 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.conf 2023-12-14T18:19:52.975 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:19:52.975 DEBUG:teuthology.orchestra.run.smithi160:> dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:19:53.034 INFO:tasks.cephadm:Adding host smithi160 to orchestrator... 2023-12-14T18:19:53.035 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch host add smithi160 2023-12-14T18:19:53.340 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:19:53.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:53 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:54.336 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.335+0000 7fe521e8d700 1 -- 172.21.15.93:0/4262438778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 msgr2=0x7fe51c0ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:54.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.335+0000 7fe521e8d700 1 --2- 172.21.15.93:0/4262438778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 0x7fe51c0ff140 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fe50c009910 tx=0x7fe50c009c20 comp rx=0 tx=0).stop 2023-12-14T18:19:54.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.335+0000 7fe521e8d700 1 -- 172.21.15.93:0/4262438778 shutdown_connections 2023-12-14T18:19:54.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.335+0000 7fe521e8d700 1 --2- 172.21.15.93:0/4262438778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 0x7fe51c0ff140 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:54.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.335+0000 7fe521e8d700 1 -- 172.21.15.93:0/4262438778 >> 172.21.15.93:0/4262438778 conn(0x7fe51c074350 msgr2=0x7fe51c074750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:54.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.335+0000 7fe521e8d700 1 -- 172.21.15.93:0/4262438778 shutdown_connections 2023-12-14T18:19:54.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.336+0000 7fe521e8d700 1 -- 172.21.15.93:0/4262438778 wait complete. 2023-12-14T18:19:54.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.336+0000 7fe521e8d700 1 Processor -- start 2023-12-14T18:19:54.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.336+0000 7fe521e8d700 1 -- start start 2023-12-14T18:19:54.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.336+0000 7fe521e8d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 0x7fe51c197500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:54.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe521e8d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe51c197a40 con 0x7fe51c0fed70 2023-12-14T18:19:54.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe51b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 0x7fe51c197500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:54.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe51b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 0x7fe51c197500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39618/0 (socket says 172.21.15.93:39618) 2023-12-14T18:19:54.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe51b7fe700 1 -- 172.21.15.93:0/4014262530 learned_addr learned my addr 172.21.15.93:0/4014262530 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe51b7fe700 1 -- 172.21.15.93:0/4014262530 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe50c0095f0 con 0x7fe51c0fed70 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe51b7fe700 1 --2- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 0x7fe51c197500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe50c004f40 tx=0x7fe50c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe518ff9700 1 -- 172.21.15.93:0/4014262530 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe50c01d070 con 0x7fe51c0fed70 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe518ff9700 1 -- 172.21.15.93:0/4014262530 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe50c004370 con 0x7fe51c0fed70 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe518ff9700 1 -- 172.21.15.93:0/4014262530 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fe50c0039b0 con 0x7fe51c0fed70 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe51c10a330 con 0x7fe51c0fed70 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.337+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe51c10a1a0 con 0x7fe51c0fed70 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.338+0000 7fe518ff9700 1 -- 172.21.15.93:0/4014262530 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fe50c003c20 con 0x7fe51c0fed70 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.338+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe51c10a4c0 con 0x7fe51c0fed70 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.338+0000 7fe518ff9700 1 --2- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe5040388c0 0x7fe50403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.338+0000 7fe518ff9700 1 -- 172.21.15.93:0/4014262530 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fe50c04c970 con 0x7fe51c0fed70 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.338+0000 7fe51affd700 1 --2- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe5040388c0 0x7fe50403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:54.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.339+0000 7fe51affd700 1 --2- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe5040388c0 0x7fe50403ad70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe510006fd0 tx=0x7fe510006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:54.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.343+0000 7fe518ff9700 1 -- 172.21.15.93:0/4014262530 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe50c029360 con 0x7fe51c0fed70 2023-12-14T18:19:54.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:54.540+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}) v1 -- 0x7fe51c197f80 con 0x7fe5040388c0 2023-12-14T18:19:54.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:54 smithi093 ceph-mon[42070]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:19:55.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi097", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: Updating smithi097:/etc/ceph/ceph.conf 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi160", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: Deploying cephadm binary to smithi160 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:19:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:55.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:55.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:56.552 INFO:teuthology.orchestra.run.smithi093.stdout:Added host 'smithi160' with addr '172.21.15.160' 2023-12-14T18:19:56.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.551+0000 7fe518ff9700 1 -- 172.21.15.93:0/4014262530 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe51c197f80 con 0x7fe5040388c0 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe5040388c0 msgr2=0x7fe50403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 --2- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe5040388c0 0x7fe50403ad70 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fe510006fd0 tx=0x7fe510006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 msgr2=0x7fe51c197500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 --2- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 0x7fe51c197500 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fe50c004f40 tx=0x7fe50c005020 comp rx=0 tx=0).stop 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 shutdown_connections 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 --2- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe5040388c0 0x7fe50403ad70 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 --2- 172.21.15.93:0/4014262530 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe51c0fed70 0x7fe51c197500 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 >> 172.21.15.93:0/4014262530 conn(0x7fe51c074350 msgr2=0x7fe51c106d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 shutdown_connections 2023-12-14T18:19:56.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:56.554+0000 7fe521e8d700 1 -- 172.21.15.93:0/4014262530 wait complete. 2023-12-14T18:19:56.922 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch host ls --format=json 2023-12-14T18:19:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:19:57.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:56 smithi093 ceph-mon[42070]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:19:57.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:56 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:57.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:56 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:19:58.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.202+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/872405540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 msgr2=0x7f6e581011c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:58.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.202+0000 7f6e5d3d5700 1 --2- 172.21.15.93:0/872405540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 0x7f6e581011c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6e48009910 tx=0x7f6e48009c20 comp rx=0 tx=0).stop 2023-12-14T18:19:58.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.202+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/872405540 shutdown_connections 2023-12-14T18:19:58.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.202+0000 7f6e5d3d5700 1 --2- 172.21.15.93:0/872405540 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 0x7f6e581011c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:58.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.202+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/872405540 >> 172.21.15.93:0/872405540 conn(0x7f6e58074930 msgr2=0x7f6e58074d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:58.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.202+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/872405540 shutdown_connections 2023-12-14T18:19:58.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.203+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/872405540 wait complete. 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.203+0000 7f6e5d3d5700 1 Processor -- start 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.203+0000 7f6e5d3d5700 1 -- start start 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.203+0000 7f6e5d3d5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 0x7f6e581951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.203+0000 7f6e5d3d5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e58195700 con 0x7f6e58100df0 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.203+0000 7f6e56ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 0x7f6e581951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.204+0000 7f6e56ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 0x7f6e581951c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39646/0 (socket says 172.21.15.93:39646) 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.204+0000 7f6e56ffd700 1 -- 172.21.15.93:0/1723038048 learned_addr learned my addr 172.21.15.93:0/1723038048 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.204+0000 7f6e56ffd700 1 -- 172.21.15.93:0/1723038048 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e480095f0 con 0x7f6e58100df0 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.204+0000 7f6e56ffd700 1 --2- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 0x7f6e581951c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006010 tx=0x7f6e48004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.204+0000 7f6e3ffff700 1 -- 172.21.15.93:0/1723038048 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e4801d070 con 0x7f6e58100df0 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.204+0000 7f6e3ffff700 1 -- 172.21.15.93:0/1723038048 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e480044a0 con 0x7f6e58100df0 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.204+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e5810a190 con 0x7f6e58100df0 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.204+0000 7f6e3ffff700 1 -- 172.21.15.93:0/1723038048 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6e48003a70 con 0x7f6e58100df0 2023-12-14T18:19:58.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.204+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5810a000 con 0x7f6e58100df0 2023-12-14T18:19:58.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.205+0000 7f6e3ffff700 1 -- 172.21.15.93:0/1723038048 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f6e48003cb0 con 0x7f6e58100df0 2023-12-14T18:19:58.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.205+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5810a320 con 0x7f6e58100df0 2023-12-14T18:19:58.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.205+0000 7f6e3ffff700 1 --2- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e400388c0 0x7f6e4003ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:19:58.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.205+0000 7f6e3ffff700 1 -- 172.21.15.93:0/1723038048 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f6e4804c690 con 0x7f6e58100df0 2023-12-14T18:19:58.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.205+0000 7f6e567fc700 1 --2- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e400388c0 0x7f6e4003ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:19:58.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.206+0000 7f6e567fc700 1 --2- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e400388c0 0x7f6e4003ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006fd0 tx=0x7f6e4c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:19:58.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.210+0000 7f6e3ffff700 1 -- 172.21.15.93:0/1723038048 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e48029770 con 0x7f6e58100df0 2023-12-14T18:19:58.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:57 smithi093 ceph-mon[42070]: Added host smithi160 2023-12-14T18:19:58.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:58.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:19:58.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.403+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6e581073e0 con 0x7f6e400388c0 2023-12-14T18:19:58.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.404+0000 7f6e3ffff700 1 -- 172.21.15.93:0/1723038048 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+239 (secure 0 0 0) 0x7f6e581073e0 con 0x7f6e400388c0 2023-12-14T18:19:58.405 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:19:58.405 INFO:teuthology.orchestra.run.smithi093.stdout:[{"addr": "172.21.15.93", "hostname": "smithi093", "labels": [], "status": ""}, {"addr": "172.21.15.97", "hostname": "smithi097", "labels": [], "status": ""}, {"addr": "172.21.15.160", "hostname": "smithi160", "labels": [], "status": ""}] 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e400388c0 msgr2=0x7f6e4003ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 --2- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e400388c0 0x7f6e4003ad70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c006fd0 tx=0x7f6e4c006e40 comp rx=0 tx=0).stop 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 msgr2=0x7f6e581951c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 --2- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 0x7f6e581951c0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f6e48006010 tx=0x7f6e48004ff0 comp rx=0 tx=0).stop 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 shutdown_connections 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 --2- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e400388c0 0x7f6e4003ad70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 --2- 172.21.15.93:0/1723038048 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e58100df0 0x7f6e581951c0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 >> 172.21.15.93:0/1723038048 conn(0x7f6e58074930 msgr2=0x7f6e58104610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 shutdown_connections 2023-12-14T18:19:58.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:19:58.406+0000 7f6e5d3d5700 1 -- 172.21.15.93:0/1723038048 wait complete. 2023-12-14T18:19:58.813 INFO:tasks.cephadm:Setting crush tunables to default 2023-12-14T18:19:58.813 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd crush tunables default 2023-12-14T18:19:59.118 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:19:59.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:59 smithi093 ceph-mon[42070]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:19:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:19:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:00.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.086+0000 7f5cd41c9700 1 -- 172.21.15.93:0/1404106916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 msgr2=0x7f5ccc1013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:00.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.086+0000 7f5cd41c9700 1 --2- 172.21.15.93:0/1404106916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 0x7f5ccc1013b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc009910 tx=0x7f5cbc009c20 comp rx=0 tx=0).stop 2023-12-14T18:20:00.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.086+0000 7f5cd41c9700 1 -- 172.21.15.93:0/1404106916 shutdown_connections 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.086+0000 7f5cd41c9700 1 --2- 172.21.15.93:0/1404106916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 0x7f5ccc1013b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.086+0000 7f5cd41c9700 1 -- 172.21.15.93:0/1404106916 >> 172.21.15.93:0/1404106916 conn(0x7f5ccc0fa880 msgr2=0x7f5ccc0fcc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.086+0000 7f5cd41c9700 1 -- 172.21.15.93:0/1404106916 shutdown_connections 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.086+0000 7f5cd41c9700 1 -- 172.21.15.93:0/1404106916 wait complete. 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd41c9700 1 Processor -- start 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd41c9700 1 -- start start 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd41c9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 0x7f5ccc195180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd41c9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ccc1956c0 con 0x7f5ccc100fe0 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd1f65700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 0x7f5ccc195180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd1f65700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 0x7f5ccc195180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39656/0 (socket says 172.21.15.93:39656) 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd1f65700 1 -- 172.21.15.93:0/2848367814 learned_addr learned my addr 172.21.15.93:0/2848367814 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd1f65700 1 -- 172.21.15.93:0/2848367814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc0095f0 con 0x7f5ccc100fe0 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd1f65700 1 --2- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 0x7f5ccc195180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc004f40 tx=0x7f5cbc005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cc2ffd700 1 -- 172.21.15.93:0/2848367814 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cbc01d070 con 0x7f5ccc100fe0 2023-12-14T18:20:00.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cc2ffd700 1 -- 172.21.15.93:0/2848367814 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5cbc004370 con 0x7f5ccc100fe0 2023-12-14T18:20:00.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ccc10a2e0 con 0x7f5ccc100fe0 2023-12-14T18:20:00.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.087+0000 7f5cc2ffd700 1 -- 172.21.15.93:0/2848367814 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5cbc0039b0 con 0x7f5ccc100fe0 2023-12-14T18:20:00.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.088+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ccc10a150 con 0x7f5ccc100fe0 2023-12-14T18:20:00.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.088+0000 7f5cc2ffd700 1 -- 172.21.15.93:0/2848367814 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f5cbc003c70 con 0x7f5ccc100fe0 2023-12-14T18:20:00.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.088+0000 7f5cc2ffd700 1 --2- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5cb8038870 0x7f5cb803ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:00.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.088+0000 7f5cc2ffd700 1 -- 172.21.15.93:0/2848367814 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f5cbc04c5e0 con 0x7f5ccc100fe0 2023-12-14T18:20:00.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.088+0000 7f5cd1764700 1 --2- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5cb8038870 0x7f5cb803ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:00.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.089+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ccc10a470 con 0x7f5ccc100fe0 2023-12-14T18:20:00.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.089+0000 7f5cd1764700 1 --2- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5cb8038870 0x7f5cb803ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8006fd0 tx=0x7f5cc8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:00.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.093+0000 7f5cc2ffd700 1 -- 172.21.15.93:0/2848367814 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5cbc026020 con 0x7f5ccc100fe0 2023-12-14T18:20:00.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:00.291+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f5ccc062120 con 0x7f5ccc100fe0 2023-12-14T18:20:00.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:00 smithi093 ceph-mon[42070]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:20:01.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.273+0000 7f5cc2ffd700 1 -- 172.21.15.93:0/2848367814 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f5cbc02b440 con 0x7f5ccc100fe0 2023-12-14T18:20:01.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5cb8038870 msgr2=0x7f5cb803ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:01.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 --2- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5cb8038870 0x7f5cb803ad20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f5cc8006fd0 tx=0x7f5cc8006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:01.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 msgr2=0x7f5ccc195180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:01.276 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 --2- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 0x7f5ccc195180 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f5cbc004f40 tx=0x7f5cbc005020 comp rx=0 tx=0).stop 2023-12-14T18:20:01.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 shutdown_connections 2023-12-14T18:20:01.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 --2- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5cb8038870 0x7f5cb803ad20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:01.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 --2- 172.21.15.93:0/2848367814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ccc100fe0 0x7f5ccc195180 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:01.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 >> 172.21.15.93:0/2848367814 conn(0x7f5ccc0fa880 msgr2=0x7f5ccc104630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:01.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 shutdown_connections 2023-12-14T18:20:01.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:01.275+0000 7f5cd41c9700 1 -- 172.21.15.93:0/2848367814 wait complete. 2023-12-14T18:20:01.277 INFO:teuthology.orchestra.run.smithi093.stderr:adjusted tunables profile to default 2023-12-14T18:20:01.553 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi160", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: Updating smithi160:/etc/ceph/ceph.conf 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2848367814' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:01.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:01.638 INFO:tasks.cephadm:Adding mon.a on smithi093 2023-12-14T18:20:01.638 INFO:tasks.cephadm:Adding mon.b on smithi097 2023-12-14T18:20:01.638 INFO:tasks.cephadm:Adding mon.c on smithi160 2023-12-14T18:20:01.638 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch apply mon '3;smithi093:172.21.15.93=a;smithi097:172.21.15.97=b;smithi160:172.21.15.160=c' 2023-12-14T18:20:01.975 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:02.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:02 smithi093 ceph-mon[42070]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:02.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:02 smithi093 ceph-mon[42070]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:20:02.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:02 smithi093 ceph-mon[42070]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:20:02.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:02 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2848367814' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2023-12-14T18:20:02.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:02 smithi093 ceph-mon[42070]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:20:02.967 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.966+0000 7fcd18689700 1 -- 172.21.15.160:0/1383748660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 msgr2=0x7fcd100ff140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:02.968 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.966+0000 7fcd18689700 1 --2- 172.21.15.160:0/1383748660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 0x7fcd100ff140 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c009910 tx=0x7fcd0c009c20 comp rx=0 tx=0).stop 2023-12-14T18:20:02.968 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.966+0000 7fcd18689700 1 -- 172.21.15.160:0/1383748660 shutdown_connections 2023-12-14T18:20:02.968 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.966+0000 7fcd18689700 1 --2- 172.21.15.160:0/1383748660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 0x7fcd100ff140 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:02.968 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.966+0000 7fcd18689700 1 -- 172.21.15.160:0/1383748660 >> 172.21.15.160:0/1383748660 conn(0x7fcd10074350 msgr2=0x7fcd10074750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:02.968 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.966+0000 7fcd18689700 1 -- 172.21.15.160:0/1383748660 shutdown_connections 2023-12-14T18:20:02.968 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.966+0000 7fcd18689700 1 -- 172.21.15.160:0/1383748660 wait complete. 2023-12-14T18:20:02.970 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.967+0000 7fcd18689700 1 Processor -- start 2023-12-14T18:20:02.970 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.967+0000 7fcd18689700 1 -- start start 2023-12-14T18:20:02.970 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.967+0000 7fcd18689700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 0x7fcd10195260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:02.970 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.967+0000 7fcd18689700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd101957a0 con 0x7fcd100fed70 2023-12-14T18:20:02.970 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd16425700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 0x7fcd10195260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:02.970 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd16425700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 0x7fcd10195260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.160:46178/0 (socket says 172.21.15.160:46178) 2023-12-14T18:20:02.970 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd16425700 1 -- 172.21.15.160:0/2836053168 learned_addr learned my addr 172.21.15.160:0/2836053168 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:20:02.970 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd16425700 1 -- 172.21.15.160:0/2836053168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd0c0095f0 con 0x7fcd100fed70 2023-12-14T18:20:02.970 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd16425700 1 --2- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 0x7fcd10195260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c004f40 tx=0x7fcd0c005020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:02.971 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd077fe700 1 -- 172.21.15.160:0/2836053168 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd0c01d070 con 0x7fcd100fed70 2023-12-14T18:20:02.971 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1010a330 con 0x7fcd100fed70 2023-12-14T18:20:02.971 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd077fe700 1 -- 172.21.15.160:0/2836053168 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcd0c004370 con 0x7fcd100fed70 2023-12-14T18:20:02.971 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd077fe700 1 -- 172.21.15.160:0/2836053168 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fcd0c0039b0 con 0x7fcd100fed70 2023-12-14T18:20:02.971 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.968+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1010a1a0 con 0x7fcd100fed70 2023-12-14T18:20:02.972 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.969+0000 7fcd077fe700 1 -- 172.21.15.160:0/2836053168 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fcd0c003c20 con 0x7fcd100fed70 2023-12-14T18:20:02.972 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.969+0000 7fcd077fe700 1 --2- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fccfc0388c0 0x7fccfc03ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:02.972 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.969+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1010a4c0 con 0x7fcd100fed70 2023-12-14T18:20:02.972 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.969+0000 7fcd077fe700 1 -- 172.21.15.160:0/2836053168 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fcd0c04c560 con 0x7fcd100fed70 2023-12-14T18:20:02.972 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.970+0000 7fcd15c24700 1 --2- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fccfc0388c0 0x7fccfc03ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:02.972 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.970+0000 7fcd15c24700 1 --2- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fccfc0388c0 0x7fccfc03ad70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd00006fd0 tx=0x7fcd00006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:02.976 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:02.975+0000 7fcd077fe700 1 -- 172.21.15.160:0/2836053168 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcd0c026020 con 0x7fcd100fed70 2023-12-14T18:20:03.174 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.173+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi097:172.21.15.97=b;smithi160:172.21.15.160=c", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd101076b0 con 0x7fccfc0388c0 2023-12-14T18:20:03.181 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.180+0000 7fcd077fe700 1 -- 172.21.15.160:0/2836053168 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fcd101076b0 con 0x7fccfc0388c0 2023-12-14T18:20:03.182 INFO:teuthology.orchestra.run.smithi160.stdout:Scheduled mon update... 2023-12-14T18:20:03.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fccfc0388c0 msgr2=0x7fccfc03ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:03.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 --2- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fccfc0388c0 0x7fccfc03ad70 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fcd00006fd0 tx=0x7fcd00006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:03.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 msgr2=0x7fcd10195260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:03.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 --2- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 0x7fcd10195260 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c004f40 tx=0x7fcd0c005020 comp rx=0 tx=0).stop 2023-12-14T18:20:03.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 shutdown_connections 2023-12-14T18:20:03.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 --2- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fccfc0388c0 0x7fccfc03ad70 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:03.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 --2- 172.21.15.160:0/2836053168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcd100fed70 0x7fcd10195260 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:03.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 >> 172.21.15.160:0/2836053168 conn(0x7fcd10074350 msgr2=0x7fcd10104930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:03.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 shutdown_connections 2023-12-14T18:20:03.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:03.182+0000 7fcd18689700 1 -- 172.21.15.160:0/2836053168 wait complete. 2023-12-14T18:20:03.564 DEBUG:teuthology.orchestra.run.smithi097:mon.b> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.b.service 2023-12-14T18:20:03.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:03 smithi093 ceph-mon[42070]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:03.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:03 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:03.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:03 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:03.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:03 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:03.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:03 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:03.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:03 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:03.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:03 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:03.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:03 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:03.568 DEBUG:teuthology.orchestra.run.smithi160:mon.c> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.c.service 2023-12-14T18:20:03.570 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-14T18:20:03.570 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-14T18:20:03.602 INFO:journalctl@ceph.mon.c.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:20:03.604 INFO:journalctl@ceph.mon.b.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:20:04.206 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:04.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:04 smithi093 ceph-mon[42070]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi097:172.21.15.97=b;smithi160:172.21.15.160=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:04.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:04 smithi093 ceph-mon[42070]: Saving service mon spec with placement smithi093:172.21.15.93=a;smithi097:172.21.15.97=b;smithi160:172.21.15.160=c;count:3 2023-12-14T18:20:04.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:04 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:04.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:04 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:04.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:04 smithi093 ceph-mon[42070]: Deploying daemon mon.c on smithi160 2023-12-14T18:20:05.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:05 smithi093 ceph-mon[42070]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:05.707 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.704+0000 7f95d0f01700 1 -- 172.21.15.160:0/3836434909 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 msgr2=0x7f95cc06bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.704+0000 7f95d0f01700 1 --2- 172.21.15.160:0/3836434909 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 0x7f95cc06bcb0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f95bc009910 tx=0x7f95bc009c20 comp rx=0 tx=0).stop 2023-12-14T18:20:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.705+0000 7f95d0f01700 1 -- 172.21.15.160:0/3836434909 shutdown_connections 2023-12-14T18:20:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.705+0000 7f95d0f01700 1 --2- 172.21.15.160:0/3836434909 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 0x7f95cc06bcb0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.705+0000 7f95d0f01700 1 -- 172.21.15.160:0/3836434909 >> 172.21.15.160:0/3836434909 conn(0x7f95cc074350 msgr2=0x7f95cc074750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.705+0000 7f95d0f01700 1 -- 172.21.15.160:0/3836434909 shutdown_connections 2023-12-14T18:20:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.705+0000 7f95d0f01700 1 -- 172.21.15.160:0/3836434909 wait complete. 2023-12-14T18:20:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.705+0000 7f95d0f01700 1 Processor -- start 2023-12-14T18:20:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.705+0000 7f95d0f01700 1 -- start start 2023-12-14T18:20:05.708 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.706+0000 7f95d0f01700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 0x7f95cc1a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:05.709 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.706+0000 7f95d0f01700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95cc1a3e80 con 0x7f95cc06b8e0 2023-12-14T18:20:05.709 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.706+0000 7f95ca59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 0x7f95cc1a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:05.709 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.706+0000 7f95ca59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 0x7f95cc1a3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.160:46196/0 (socket says 172.21.15.160:46196) 2023-12-14T18:20:05.709 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.706+0000 7f95ca59c700 1 -- 172.21.15.160:0/3125149256 learned_addr learned my addr 172.21.15.160:0/3125149256 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:20:05.709 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.706+0000 7f95ca59c700 1 -- 172.21.15.160:0/3125149256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95bc0095f0 con 0x7f95cc06b8e0 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.707+0000 7f95ca59c700 1 --2- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 0x7f95cc1a3940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0092e0 tx=0x7f95bc004ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.707+0000 7f95bb7fe700 1 -- 172.21.15.160:0/3125149256 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95bc01d070 con 0x7f95cc06b8e0 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.707+0000 7f95bb7fe700 1 -- 172.21.15.160:0/3125149256 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95bc00bc10 con 0x7f95cc06b8e0 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.707+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95cc111120 con 0x7f95cc06b8e0 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.707+0000 7f95bb7fe700 1 -- 172.21.15.160:0/3125149256 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f95bc003930 con 0x7f95cc06b8e0 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.707+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95cc110f90 con 0x7f95cc06b8e0 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.708+0000 7f95bb7fe700 1 -- 172.21.15.160:0/3125149256 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f95bc003b60 con 0x7f95cc06b8e0 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.708+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95cc1112b0 con 0x7f95cc06b8e0 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.708+0000 7f95bb7fe700 1 --2- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95b40388c0 0x7f95b403ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:05.711 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.708+0000 7f95bb7fe700 1 -- 172.21.15.160:0/3125149256 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f95bc04c410 con 0x7f95cc06b8e0 2023-12-14T18:20:05.712 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.708+0000 7f95c9d9b700 1 --2- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95b40388c0 0x7f95b403ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:05.712 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.709+0000 7f95c9d9b700 1 --2- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95b40388c0 0x7f95b403ad70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:05.714 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:05.713+0000 7f95bb7fe700 1 -- 172.21.15.160:0/3125149256 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f95bc029950 con 0x7f95cc06b8e0 2023-12-14T18:20:05.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:05 smithi160 systemd[1]: Starting Ceph mon.c for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:20:06.009 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.007+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f95cc19efe0 con 0x7f95cc06b8e0 2023-12-14T18:20:06.009 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.008+0000 7f95bb7fe700 1 -- 172.21.15.160:0/3125149256 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+736 (secure 0 0 0) 0x7f95bc026070 con 0x7f95cc06b8e0 2023-12-14T18:20:06.009 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:20:06.009 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":1,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","modified":"2023-12-14T18:18:43.750667Z","created":"2023-12-14T18:18:43.750667Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2023-12-14T18:20:06.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95b40388c0 msgr2=0x7f95b403ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:06.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 --2- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95b40388c0 0x7f95b403ad70 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f95c0006fd0 tx=0x7f95c0006e40 comp rx=0 tx=0).stop 2023-12-14T18:20:06.011 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 msgr2=0x7f95cc1a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:06.012 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 --2- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 0x7f95cc1a3940 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f95bc0092e0 tx=0x7f95bc004ff0 comp rx=0 tx=0).stop 2023-12-14T18:20:06.012 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 shutdown_connections 2023-12-14T18:20:06.012 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 --2- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95b40388c0 0x7f95b403ad70 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:06.012 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 --2- 172.21.15.160:0/3125149256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95cc06b8e0 0x7f95cc1a3940 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:06.012 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 >> 172.21.15.160:0/3125149256 conn(0x7f95cc074350 msgr2=0x7f95cc0fe290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:06.012 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 shutdown_connections 2023-12-14T18:20:06.012 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:06.010+0000 7f95d0f01700 1 -- 172.21.15.160:0/3125149256 wait complete. 2023-12-14T18:20:06.012 INFO:teuthology.orchestra.run.smithi160.stderr:dumped monmap epoch 1 2023-12-14T18:20:06.302 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: set uid:gid to 167:167 (ceph:ceph) 2023-12-14T18:20:06.302 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev), process ceph-mon, pid 7 2023-12-14T18:20:06.302 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: pidfile_write: ignore empty --pid-file 2023-12-14T18:20:06.302 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: load: jerasure load: lrc 2023-12-14T18:20:06.302 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: RocksDB version: 7.9.2 2023-12-14T18:20:06.302 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Git sha 0 2023-12-14T18:20:06.302 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Compile date 2023-12-13 16:19:48 2023-12-14T18:20:06.302 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: DB SUMMARY 2023-12-14T18:20:06.302 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: DB Session ID: KRMNLCX3PYXRUQHYQN84 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: CURRENT file: CURRENT 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: IDENTITY file: IDENTITY 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.error_if_exists: 0 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.create_if_missing: 0 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.paranoid_checks: 1 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.env: 0x5559dfea8720 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.fs: PosixFileSystem 2023-12-14T18:20:06.303 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.info_log: 0x5559e29f73c0 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_file_opening_threads: 16 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.statistics: (nil) 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.use_fsync: 0 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_log_file_size: 0 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.keep_log_file_num: 1000 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.recycle_log_file_num: 0 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.allow_fallocate: 1 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.allow_mmap_reads: 0 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.allow_mmap_writes: 0 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.use_direct_reads: 0 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-14T18:20:06.304 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.create_missing_column_families: 0 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.db_log_dir: 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.wal_dir: 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.advise_random_on_open: 1 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.db_write_buffer_size: 0 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.write_buffer_manager: 0x5559e1c683c0 2023-12-14T18:20:06.305 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.rate_limiter: (nil) 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.wal_recovery_mode: 2 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.enable_thread_tracking: 0 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.enable_pipelined_write: 0 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.unordered_write: 0 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-14T18:20:06.306 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.row_cache: None 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.wal_filter: None 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.allow_ingest_behind: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.two_write_queues: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.manual_wal_flush: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.wal_compression: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.atomic_flush: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.log_readahead_size: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.best_efforts_recovery: 0 2023-12-14T18:20:06.307 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.allow_data_in_errors: 0 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.db_host_id: __hostname__ 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_background_jobs: 2 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_background_compactions: -1 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_subcompactions: 1 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-14T18:20:06.308 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_total_wal_size: 0 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_open_files: -1 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bytes_per_sync: 0 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_readahead_size: 0 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_background_flushes: -1 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Compression algorithms supported: 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-14T18:20:06.309 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: kZSTD supported: 0 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: kXpressCompression supported: 0 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: kLZ4HCCompression supported: 1 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: kZlibCompression supported: 1 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: kSnappyCompression supported: 1 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: kLZ4Compression supported: 1 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: kBZip2Compression supported: 0 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.merge_operator: 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_filter: None 2023-12-14T18:20:06.310 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_filter_factory: None 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.sst_partitioner_factory: None 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5559e29f74e0) 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: cache_index_and_filter_blocks: 1 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: pin_top_level_index_and_filter: 1 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: index_type: 0 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: data_block_index_type: 0 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: index_shortening: 1 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: checksum: 4 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: no_block_cache: 0 2023-12-14T18:20:06.311 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_cache: 0x5559e1d091f0 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_cache_name: BinnedLRUCache 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_cache_options: 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: capacity : 536870912 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: num_shard_bits : 4 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: strict_capacity_limit : 0 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: high_pri_pool_ratio: 0.000 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_cache_compressed: (nil) 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: persistent_cache: (nil) 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_size: 4096 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_size_deviation: 10 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_restart_interval: 16 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: index_block_restart_interval: 1 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: metadata_block_size: 4096 2023-12-14T18:20:06.312 INFO:journalctl@ceph.mon.c.smithi160.stdout: partition_filters: 0 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: use_delta_encoding: 1 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: filter_policy: bloomfilter 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: whole_key_filtering: 1 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: verify_compression: 0 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: read_amp_bytes_per_bit: 0 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: format_version: 5 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: enable_index_compression: 1 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: block_align: 0 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: max_auto_readahead_size: 262144 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: prepopulate_block_cache: 0 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: initial_auto_readahead_size: 8192 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout: num_file_reads_for_auto_readahead: 2 2023-12-14T18:20:06.313 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.write_buffer_size: 33554432 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_write_buffer_number: 2 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression: NoCompression 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression: Disabled 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.prefix_extractor: nullptr 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.num_levels: 7 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:20:06.314 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression_opts.level: 32767 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression_opts.strategy: 0 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression_opts.enabled: false 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-14T18:20:06.315 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.target_file_size_base: 67108864 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-14T18:20:06.316 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.arena_block_size: 1048576 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.disable_auto_compactions: 0 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-14T18:20:06.317 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.table_properties_collectors: 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.inplace_update_support: 0 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.bloom_locality: 0 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.max_successive_merges: 0 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.paranoid_file_checks: 0 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.force_consistency_checks: 1 2023-12-14T18:20:06.318 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.report_bg_io_stats: 0 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.ttl: 2592000 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.enable_blob_files: false 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.min_blob_size: 0 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.blob_file_size: 268435456 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.blob_file_starting_level: 0 2023-12-14T18:20:06.319 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 969d747e-3b40-4c67-b53e-10da600ac53c 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578006204583, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578006205006, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1622, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702578006, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "969d747e-3b40-4c67-b53e-10da600ac53c", "db_session_id": "KRMNLCX3PYXRUQHYQN84", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578006205073, "job": 1, "event": "recovery_finished"} 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5559e1da6000 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: DB pointer 0x5559e1d92000 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout: ** DB Stats ** 2023-12-14T18:20:06.320 INFO:journalctl@ceph.mon.c.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: ** Compaction Stats [default] ** 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: L0 1/0 1.58 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: Sum 1/0 1.58 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2023-12-14T18:20:06.321 INFO:journalctl@ceph.mon.c.smithi160.stdout: ** Compaction Stats [default] ** 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.8 0.00 0.00 1 0.000 0 0 0.0 0.0 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:20:06.322 INFO:journalctl@ceph.mon.c.smithi160.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout: Block cache BinnedLRUCache@0x5559e1d091f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c does not exist in monmap, will attempt to join an existing cluster 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: using public_addr v2:172.21.15.160:0/0 -> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: starting mon.c rank -1 at public addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] at bind addrs [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(???) e0 preinit fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 bash[43784]: da39c0c8238ff8908c5e06be5f618aafe50c0cf26f3d54e9022fa7f3375f7dbe 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 systemd[1]: Started Ceph mon.c for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).mds e1 new map 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).mds e1 print_map 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout: e1 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-14T18:20:06.323 INFO:journalctl@ceph.mon.c.smithi160.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout: legacy client fscid: -1 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout: 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout: No filesystems configured 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:06.324 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi097:172.21.15.97=b;smithi160:172.21.15.160=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: Saving service mon spec with placement smithi093:172.21.15.93=a;smithi097:172.21.15.97=b;smithi160:172.21.15.160=c;count:3 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: Deploying daemon mon.c on smithi160 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: mon.c@-1(synchronizing) e1 handle_conf_change mon_allow_pool_delete 2023-12-14T18:20:06.325 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta expand map: {default=false} 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta from 'false' to 'false' 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta expanded map: {default=false} 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta expand map: {default=info} 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta from 'info' to 'info' 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta expanded map: {default=info} 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta expand map: {default=daemon} 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta from 'daemon' to 'daemon' 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta expanded map: {default=daemon} 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta expand map: {default=debug} 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta from 'debug' to 'debug' 2023-12-14T18:20:06.326 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:06 smithi160 ceph-mon[43840]: expand_channel_meta expanded map: {default=debug} 2023-12-14T18:20:07.442 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-14T18:20:07.442 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-14T18:20:07.758 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:20:11.722 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.720+0000 7f61c1b10700 1 -- 172.21.15.160:0/6942171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61a40035c0 msgr2=0x7f61a4005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:11.722 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.720+0000 7f61c1b10700 1 --2- 172.21.15.160:0/6942171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61a40035c0 0x7f61a4005a40 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f61bc076020 tx=0x7f61ac00fe60 comp rx=0 tx=0).stop 2023-12-14T18:20:11.722 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.721+0000 7f61c1b10700 1 -- 172.21.15.160:0/6942171 shutdown_connections 2023-12-14T18:20:11.723 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.721+0000 7f61c1b10700 1 --2- 172.21.15.160:0/6942171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61a40035c0 0x7f61a4005a40 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:11.723 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.721+0000 7f61c1b10700 1 -- 172.21.15.160:0/6942171 >> 172.21.15.160:0/6942171 conn(0x7f61bc074c40 msgr2=0x7f61bc075040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:11.723 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.721+0000 7f61c1b10700 1 -- 172.21.15.160:0/6942171 shutdown_connections 2023-12-14T18:20:11.723 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.721+0000 7f61c1b10700 1 -- 172.21.15.160:0/6942171 wait complete. 2023-12-14T18:20:11.723 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.722+0000 7f61c1b10700 1 Processor -- start 2023-12-14T18:20:11.723 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61c1b10700 1 -- start start 2023-12-14T18:20:11.725 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61c1b10700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61a40035c0 0x7f61bc197460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:11.725 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61c1b10700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 0x7f61bc19be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:11.725 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61c1b10700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc197f20 con 0x7f61a40035c0 2023-12-14T18:20:11.725 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61c1b10700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61bc198090 con 0x7f61bc1979a0 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61baffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 0x7f61bc19be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61bb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61a40035c0 0x7f61bc197460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61baffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 0x7f61bc19be10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53386/0 (socket says 172.21.15.160:53386) 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61baffd700 1 -- 172.21.15.160:0/2295036082 learned_addr learned my addr 172.21.15.160:0/2295036082 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61baffd700 1 -- 172.21.15.160:0/2295036082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 msgr2=0x7f61bc19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61baffd700 1 -- 172.21.15.160:0/2295036082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 msgr2=0x7f61bc19be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61baffd700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 0x7f61bc19be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.723+0000 7f61baffd700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 0x7f61bc19be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.724+0000 7f61bb7fe700 1 -- 172.21.15.160:0/2295036082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 msgr2=0x7f61bc19be10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.724+0000 7f61bb7fe700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 0x7f61bc19be10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.724+0000 7f61bb7fe700 1 -- 172.21.15.160:0/2295036082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61ac00f8b0 con 0x7f61a40035c0 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.724+0000 7f61bb7fe700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61a40035c0 0x7f61bc197460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61bc076020 tx=0x7f61ac013c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.724+0000 7f61b8ff9700 1 -- 172.21.15.160:0/2295036082 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f61ac023070 con 0x7f61a40035c0 2023-12-14T18:20:11.726 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.724+0000 7f61b8ff9700 1 -- 172.21.15.160:0/2295036082 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f61ac004bd0 con 0x7f61a40035c0 2023-12-14T18:20:11.727 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.724+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61bc109ec0 con 0x7f61a40035c0 2023-12-14T18:20:11.727 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.724+0000 7f61b8ff9700 1 -- 172.21.15.160:0/2295036082 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7f61ac003da0 con 0x7f61a40035c0 2023-12-14T18:20:11.727 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.724+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61bc109d30 con 0x7f61a40035c0 2023-12-14T18:20:11.727 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.725+0000 7f61b8ff9700 1 -- 172.21.15.160:0/2295036082 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f61ac004060 con 0x7f61a40035c0 2023-12-14T18:20:11.727 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.725+0000 7f61b8ff9700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61a8038700 0x7f61a803abb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:11.728 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.725+0000 7f61b8ff9700 1 -- 172.21.15.160:0/2295036082 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f61ac04e6b0 con 0x7f61a40035c0 2023-12-14T18:20:11.728 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.726+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61bc10a050 con 0x7f61a40035c0 2023-12-14T18:20:11.728 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.726+0000 7f61baffd700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61a8038700 0x7f61a803abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:11.728 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.726+0000 7f61baffd700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61a8038700 0x7f61a803abb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009180 tx=0x7f61b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:11.732 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.731+0000 7f61b8ff9700 1 -- 172.21.15.160:0/2295036082 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f61ac01a970 con 0x7f61a40035c0 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: mon.a calling monitor election 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: mon.c calling monitor election 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:11.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: monmap e2: 2 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: fsmap 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: mgrmap e13: x(active, since 35s) 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: overall HEALTH_OK 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:11.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:11.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:20:11.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: mon.a calling monitor election 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: mon.c calling monitor election 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: monmap e2: 2 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: fsmap 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: mgrmap e13: x(active, since 35s) 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: overall HEALTH_OK 2023-12-14T18:20:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:11.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:11.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:11.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:11.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:12.000 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.999+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f61bc198890 con 0x7f61a40035c0 2023-12-14T18:20:12.000 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:11.999+0000 7f61b8ff9700 1 -- 172.21.15.160:0/2295036082 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1006 (secure 0 0 0) 0x7f61ac01a380 con 0x7f61a40035c0 2023-12-14T18:20:12.001 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:20:12.001 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":2,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","modified":"2023-12-14T18:20:06.227898Z","created":"2023-12-14T18:18:43.750667Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61a8038700 msgr2=0x7f61a803abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61a8038700 0x7f61a803abb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f61b0009180 tx=0x7f61b0008040 comp rx=0 tx=0).stop 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61a40035c0 msgr2=0x7f61bc197460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61a40035c0 0x7f61bc197460 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f61bc076020 tx=0x7f61ac013c90 comp rx=0 tx=0).stop 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 shutdown_connections 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61a8038700 0x7f61a803abb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61a40035c0 0x7f61bc197460 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 --2- 172.21.15.160:0/2295036082 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f61bc1979a0 0x7f61bc19be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 >> 172.21.15.160:0/2295036082 conn(0x7f61bc074c40 msgr2=0x7f61bc107de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 shutdown_connections 2023-12-14T18:20:12.003 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:12.001+0000 7f61c1b10700 1 -- 172.21.15.160:0/2295036082 wait complete. 2023-12-14T18:20:12.004 INFO:teuthology.orchestra.run.smithi160.stderr:dumped monmap epoch 2 2023-12-14T18:20:12.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:12 smithi160 ceph-mon[43840]: Deploying daemon mon.b on smithi097 2023-12-14T18:20:12.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:12 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/2295036082' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:20:12.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:12 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:12.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:12 smithi093 ceph-mon[42070]: Deploying daemon mon.b on smithi097 2023-12-14T18:20:12.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:12 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/2295036082' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:20:12.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:12 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:13.391 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2023-12-14T18:20:13.391 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph mon dump -f json 2023-12-14T18:20:13.705 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:20:13.740 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 systemd[1]: Starting Ceph mon.b for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:20:13.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:13 smithi160 ceph-mon[43840]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:13.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:13 smithi093 ceph-mon[42070]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:14.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: set uid:gid to 167:167 (ceph:ceph) 2023-12-14T18:20:14.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev), process ceph-mon, pid 7 2023-12-14T18:20:14.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: pidfile_write: ignore empty --pid-file 2023-12-14T18:20:14.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: load: jerasure load: lrc 2023-12-14T18:20:14.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: RocksDB version: 7.9.2 2023-12-14T18:20:14.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Git sha 0 2023-12-14T18:20:14.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Compile date 2023-12-13 16:19:48 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: DB SUMMARY 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: DB Session ID: OKYAHONRE77JDRT30O9V 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: CURRENT file: CURRENT 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: IDENTITY file: IDENTITY 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 636 ; 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.error_if_exists: 0 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.create_if_missing: 0 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.paranoid_checks: 1 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.flush_verify_memtable_count: 1 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.env: 0x55a342001720 2023-12-14T18:20:14.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.fs: PosixFileSystem 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.info_log: 0x55a3445893c0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_file_opening_threads: 16 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.statistics: (nil) 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.use_fsync: 0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_log_file_size: 0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_manifest_file_size: 1073741824 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.log_file_time_to_roll: 0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.keep_log_file_num: 1000 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.recycle_log_file_num: 0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.allow_fallocate: 1 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.allow_mmap_reads: 0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.allow_mmap_writes: 0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.use_direct_reads: 0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.create_missing_column_families: 0 2023-12-14T18:20:14.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.db_log_dir: 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.wal_dir: 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.table_cache_numshardbits: 6 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.WAL_ttl_seconds: 0 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.WAL_size_limit_MB: 0 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.manifest_preallocation_size: 4194304 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.is_fd_close_on_exec: 1 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.advise_random_on_open: 1 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.db_write_buffer_size: 0 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.write_buffer_manager: 0x55a3437fa3c0 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.access_hint_on_compaction_start: 1 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.random_access_max_buffer_size: 1048576 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.use_adaptive_mutex: 0 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.rate_limiter: (nil) 2023-12-14T18:20:14.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.wal_recovery_mode: 2 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.enable_thread_tracking: 0 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.enable_pipelined_write: 0 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.unordered_write: 0 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.allow_concurrent_memtable_write: 1 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.write_thread_max_yield_usec: 100 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.write_thread_slow_yield_usec: 3 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.row_cache: None 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.wal_filter: None 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.avoid_flush_during_recovery: 0 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.allow_ingest_behind: 0 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.two_write_queues: 0 2023-12-14T18:20:14.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.manual_wal_flush: 0 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.wal_compression: 0 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.atomic_flush: 0 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.persist_stats_to_disk: 0 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.write_dbid_to_manifest: 0 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.log_readahead_size: 0 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.file_checksum_gen_factory: Unknown 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.best_efforts_recovery: 0 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.allow_data_in_errors: 0 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.db_host_id: __hostname__ 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.enforce_single_del_contracts: true 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_background_jobs: 2 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_background_compactions: -1 2023-12-14T18:20:14.098 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_subcompactions: 1 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.avoid_flush_during_shutdown: 0 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.delayed_write_rate : 16777216 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_total_wal_size: 0 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.stats_dump_period_sec: 600 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.stats_persist_period_sec: 600 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.stats_history_buffer_size: 1048576 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_open_files: -1 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bytes_per_sync: 0 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.wal_bytes_per_sync: 0 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.strict_bytes_per_sync: 0 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_readahead_size: 0 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_background_flushes: -1 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Compression algorithms supported: 2023-12-14T18:20:14.099 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: kZSTDNotFinalCompression supported: 0 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: kZSTD supported: 0 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: kXpressCompression supported: 0 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: kLZ4HCCompression supported: 1 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: kZlibCompression supported: 1 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: kSnappyCompression supported: 1 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: kLZ4Compression supported: 1 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: kBZip2Compression supported: 0 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Fast CRC32 supported: Supported on x86 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: DMutex implementation: pthread_mutex_t 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2023-12-14T18:20:14.100 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.merge_operator: 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_filter: None 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_filter_factory: None 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.sst_partitioner_factory: None 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.memtable_factory: SkipListFactory 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.table_factory: BlockBasedTable 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55a3445894e0) 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout: cache_index_and_filter_blocks: 1 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout: pin_top_level_index_and_filter: 1 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout: index_type: 0 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout: data_block_index_type: 0 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout: index_shortening: 1 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout: data_block_hash_table_util_ratio: 0.750000 2023-12-14T18:20:14.101 INFO:journalctl@ceph.mon.b.smithi097.stdout: checksum: 4 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: no_block_cache: 0 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache: 0x55a34389b1f0 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache_name: BinnedLRUCache 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache_options: 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: capacity : 536870912 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: num_shard_bits : 4 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: strict_capacity_limit : 0 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: high_pri_pool_ratio: 0.000 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_cache_compressed: (nil) 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: persistent_cache: (nil) 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_size: 4096 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_size_deviation: 10 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_restart_interval: 16 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: index_block_restart_interval: 1 2023-12-14T18:20:14.102 INFO:journalctl@ceph.mon.b.smithi097.stdout: metadata_block_size: 4096 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: partition_filters: 0 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: use_delta_encoding: 1 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: filter_policy: bloomfilter 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: whole_key_filtering: 1 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: verify_compression: 0 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: read_amp_bytes_per_bit: 0 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: format_version: 5 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: enable_index_compression: 1 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: block_align: 0 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: max_auto_readahead_size: 262144 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: prepopulate_block_cache: 0 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: initial_auto_readahead_size: 8192 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout: num_file_reads_for_auto_readahead: 2 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.write_buffer_size: 33554432 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_write_buffer_number: 2 2023-12-14T18:20:14.103 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression: NoCompression 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression: Disabled 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.prefix_extractor: nullptr 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.num_levels: 7 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression_opts.level: 32767 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2023-12-14T18:20:14.104 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:20:14.105 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2023-12-14T18:20:14.105 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression_opts.enabled: false 2023-12-14T18:20:14.105 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:20:14.105 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression_opts.window_bits: -14 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression_opts.level: 32767 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression_opts.strategy: 0 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 bash[43448]: a433b41687fff9c1e52572e83b5b756e77dc210f19960c416c2be770c4463e82 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression_opts.parallel_threads: 1 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression_opts.enabled: false 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.level0_stop_writes_trigger: 36 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.target_file_size_base: 67108864 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.target_file_size_multiplier: 1 2023-12-14T18:20:14.106 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bytes_for_level_base: 268435456 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_compaction_bytes: 1677721600 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.arena_block_size: 1048576 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2023-12-14T18:20:14.107 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.disable_auto_compactions: 0 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.table_properties_collectors: 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.inplace_update_support: 0 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.inplace_update_num_locks: 10000 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.memtable_whole_key_filtering: 0 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.memtable_huge_page_size: 0 2023-12-14T18:20:14.108 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.bloom_locality: 0 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.max_successive_merges: 0 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.optimize_filters_for_hits: 0 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.paranoid_file_checks: 0 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.force_consistency_checks: 1 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.report_bg_io_stats: 0 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.ttl: 2592000 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.periodic_compaction_seconds: 0 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.preclude_last_level_data_seconds: 0 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.preserve_internal_time_seconds: 0 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.enable_blob_files: false 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.min_blob_size: 0 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.blob_file_size: 268435456 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.blob_compression_type: NoCompression 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.enable_blob_garbage_collection: false 2023-12-14T18:20:14.109 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.blob_compaction_readahead_size: 0 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.blob_file_starting_level: 0 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 4cfbd71c-e536-4635-8674-062ab8aab862 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578013809826, "job": 1, "event": "recovery_started", "wal_files": [4]} 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578013810386, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1702578013, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "4cfbd71c-e536-4635-8674-062ab8aab862", "db_session_id": "OKYAHONRE77JDRT30O9V", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: EVENT_LOG_v1 {"time_micros": 1702578013810483, "job": 1, "event": "recovery_finished"} 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55a343938000 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: DB pointer 0x55a343924000 2023-12-14T18:20:14.110 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: rocksdb: [db/db_impl/db_impl.cc:1111] 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** DB Stats ** 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** Compaction Stats [default] ** 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: L0 1/0 1.71 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Sum 1/0 1.71 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:20:14.111 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** Compaction Stats [default] ** 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 3.1 0.00 0.00 1 0.001 0 0 0.0 0.0 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: Uptime(secs): 0.0 total, 0.0 interval 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: Flush(GB): cumulative 0.000, interval 0.000 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(Total Files): cumulative 0, interval 0 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(L0 Files): cumulative 0, interval 0 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: AddFile(Keys): cumulative 0, interval 0 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: Cumulative compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: Interval compaction: 0.00 GB write, 0.49 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2023-12-14T18:20:14.112 INFO:journalctl@ceph.mon.b.smithi097.stdout: Block cache BinnedLRUCache@0x55a34389b1f0#7 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout: ** File Read Latency Histogram By Level [default] ** 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b does not exist in monmap, will attempt to join an existing cluster 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: using public_addr v2:172.21.15.97:0/0 -> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: starting mon.b rank -1 at public addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] at bind addrs [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(???) e0 preinit fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 systemd[1]: Started Ceph mon.b for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).mds e1 new map 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).mds e1 print_map 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout: e1 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout: enable_multiple, ever_enabled_multiple: 1,1 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout: legacy client fscid: -1 2023-12-14T18:20:14.113 INFO:journalctl@ceph.mon.b.smithi097.stdout: 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout: No filesystems configured 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:14.114 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi093:172.21.15.93=a;smithi097:172.21.15.97=b;smithi160:172.21.15.160=c", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: Saving service mon spec with placement smithi093:172.21.15.93=a;smithi097:172.21.15.97=b;smithi160:172.21.15.160=c;count:3 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: Deploying daemon mon.c on smithi160 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.a calling monitor election 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:14.115 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.c calling monitor election 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: monmap e2: 2 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],c=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: fsmap 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mgrmap e13: x(active, since 35s) 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: overall HEALTH_OK 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:14.116 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: Deploying daemon mon.b on smithi097 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/2295036082' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: mon.b@-1(synchronizing) e2 handle_conf_change mon_allow_pool_delete 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta expand map: {default=false} 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta from 'false' to 'false' 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta expanded map: {default=false} 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta expand map: {default=info} 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta from 'info' to 'info' 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta expanded map: {default=info} 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta expand map: {default=daemon} 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta from 'daemon' to 'daemon' 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta expanded map: {default=daemon} 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta expand map: {default=debug} 2023-12-14T18:20:14.117 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta from 'debug' to 'debug' 2023-12-14T18:20:14.118 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:13 smithi097 ceph-mon[43486]: expand_channel_meta expanded map: {default=debug} 2023-12-14T18:20:19.065 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.064+0000 7f5556b20700 1 -- 172.21.15.160:0/1169348042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55400035c0 msgr2=0x7f5540005a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:19.065 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.064+0000 7f5556b20700 1 --2- 172.21.15.160:0/1169348042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55400035c0 0x7f5540005a40 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5550075f20 tx=0x7f554400fe10 comp rx=0 tx=0).stop 2023-12-14T18:20:19.066 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.064+0000 7f5556b20700 1 -- 172.21.15.160:0/1169348042 shutdown_connections 2023-12-14T18:20:19.066 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.064+0000 7f5556b20700 1 --2- 172.21.15.160:0/1169348042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f55400035c0 0x7f5540005a40 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.066 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.064+0000 7f5556b20700 1 -- 172.21.15.160:0/1169348042 >> 172.21.15.160:0/1169348042 conn(0x7f5550074c40 msgr2=0x7f5550075040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:19.066 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.064+0000 7f5556b20700 1 -- 172.21.15.160:0/1169348042 shutdown_connections 2023-12-14T18:20:19.066 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.064+0000 7f5556b20700 1 -- 172.21.15.160:0/1169348042 wait complete. 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f5556b20700 1 Processor -- start 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f5556b20700 1 -- start start 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f5556b20700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 0x7f55501961c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f5556b20700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5550196700 0x7f5550196b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f5556b20700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555019f230 0x7f55501a1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f5556b20700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55501a1dc0 con 0x7f5550196700 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f5556b20700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55501a1f30 con 0x7f555019f230 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f5556b20700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55501a20f0 con 0x7f55400035c0 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f55548bc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 0x7f55501961c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:19.068 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f554ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5550196700 0x7f5550196b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f554ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5550196700 0x7f5550196b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.160:34380/0 (socket says 172.21.15.160:34380) 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f55548bc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 0x7f55501961c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:53398/0 (socket says 172.21.15.160:53398) 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f554ffff700 1 -- 172.21.15.160:0/3794013663 learned_addr learned my addr 172.21.15.160:0/3794013663 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.065+0000 7f55550bd700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555019f230 0x7f55501a1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f55548bc700 1 -- 172.21.15.160:0/3794013663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 msgr2=0x7f55501961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f55548bc700 1 -- 172.21.15.160:0/3794013663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 msgr2=0x7f55501961c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f55548bc700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 0x7f55501961c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f55548bc700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 0x7f55501961c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f55550bd700 1 -- 172.21.15.160:0/3794013663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555019f230 msgr2=0x7f55501a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f55550bd700 1 -- 172.21.15.160:0/3794013663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555019f230 msgr2=0x7f55501a1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f55550bd700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555019f230 0x7f55501a1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f55550bd700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555019f230 0x7f55501a1610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f554ffff700 1 -- 172.21.15.160:0/3794013663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 msgr2=0x7f55501961c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:20:19.069 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f554ffff700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 0x7f55501961c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f554ffff700 1 -- 172.21.15.160:0/3794013663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555019f230 msgr2=0x7f55501a1610 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:20:19.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f554ffff700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555019f230 0x7f55501a1610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f554ffff700 1 -- 172.21.15.160:0/3794013663 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f554400f860 con 0x7f5550196700 2023-12-14T18:20:19.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f554ffff700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5550196700 0x7f5550196b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f553c00b780 tx=0x7f553c00bb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:19.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f554dffb700 1 -- 172.21.15.160:0/3794013663 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c004230 con 0x7f5550196700 2023-12-14T18:20:19.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f554dffb700 1 -- 172.21.15.160:0/3794013663 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f553c004390 con 0x7f5550196700 2023-12-14T18:20:19.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5550106bf0 con 0x7f5550196700 2023-12-14T18:20:19.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.066+0000 7f554dffb700 1 -- 172.21.15.160:0/3794013663 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f553c011b30 con 0x7f5550196700 2023-12-14T18:20:19.070 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.067+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5550106a60 con 0x7f5550196700 2023-12-14T18:20:19.071 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.068+0000 7f554dffb700 1 -- 172.21.15.160:0/3794013663 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f553c011cd0 con 0x7f5550196700 2023-12-14T18:20:19.071 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.068+0000 7f554dffb700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5534038d40 0x7f553403b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:19.071 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.068+0000 7f554dffb700 1 -- 172.21.15.160:0/3794013663 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f553c04aec0 con 0x7f5550196700 2023-12-14T18:20:19.071 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.068+0000 7f55548bc700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5534038d40 0x7f553403b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:19.071 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.068+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5550106d80 con 0x7f5550196700 2023-12-14T18:20:19.071 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.069+0000 7f55548bc700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5534038d40 0x7f553403b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5544010010 tx=0x7f5544005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:19.074 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.073+0000 7f554dffb700 1 -- 172.21.15.160:0/3794013663 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f553c01a400 con 0x7f5550196700 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: mon.a calling monitor election 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: mon.c calling monitor election 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: mon.b calling monitor election 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],c=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: fsmap 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: mgrmap e13: x(active, since 43s) 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: overall HEALTH_OK 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:19.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:19.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: mon.a calling monitor election 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: mon.c calling monitor election 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: mon.b calling monitor election 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-14T18:20:19.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],c=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} 2023-12-14T18:20:19.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: fsmap 2023-12-14T18:20:19.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:20:19.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: mgrmap e13: x(active, since 43s) 2023-12-14T18:20:19.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: overall HEALTH_OK 2023-12-14T18:20:19.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:19.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:19.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: mon.a calling monitor election 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: mon.c calling monitor election 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: mon.b calling monitor election 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: monmap e3: 3 mons at {a=[v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0],b=[v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0],c=[v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0]} removed_ranks: {} 2023-12-14T18:20:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: fsmap 2023-12-14T18:20:19.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: osdmap e4: 0 total, 0 up, 0 in 2023-12-14T18:20:19.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: mgrmap e13: x(active, since 43s) 2023-12-14T18:20:19.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: overall HEALTH_OK 2023-12-14T18:20:19.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:19.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:19.356 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.355+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f55501a2ca0 con 0x7f5550196700 2023-12-14T18:20:19.357 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.356+0000 7f554dffb700 1 -- 172.21.15.160:0/3794013663 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1272 (secure 0 0 0) 0x7f553c00f2e0 con 0x7f5550196700 2023-12-14T18:20:19.357 INFO:teuthology.orchestra.run.smithi160.stdout: 2023-12-14T18:20:19.357 INFO:teuthology.orchestra.run.smithi160.stdout:{"epoch":3,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","modified":"2023-12-14T18:20:13.835733Z","created":"2023-12-14T18:18:43.750667Z","min_mon_release":18,"min_mon_release_name":"reef","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:3300","nonce":0},{"type":"v1","addr":"172.21.15.93:6789","nonce":0}]},"addr":"172.21.15.93:6789/0","public_addr":"172.21.15.93:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:3300","nonce":0},{"type":"v1","addr":"172.21.15.160:6789","nonce":0}]},"addr":"172.21.15.160:6789/0","public_addr":"172.21.15.160:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:3300","nonce":0},{"type":"v1","addr":"172.21.15.97:6789","nonce":0}]},"addr":"172.21.15.97:6789/0","public_addr":"172.21.15.97:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2023-12-14T18:20:19.359 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5534038d40 msgr2=0x7f553403b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5534038d40 0x7f553403b1f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5544010010 tx=0x7f5544005a90 comp rx=0 tx=0).stop 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5550196700 msgr2=0x7f5550196b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5550196700 0x7f5550196b70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f553c00b780 tx=0x7f553c00bb40 comp rx=0 tx=0).stop 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 shutdown_connections 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5534038d40 0x7f553403b1f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f555019f230 0x7f55501a1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f55400035c0 0x7f55501961c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 --2- 172.21.15.160:0/3794013663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5550196700 0x7f5550196b70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 >> 172.21.15.160:0/3794013663 conn(0x7f5550074c40 msgr2=0x7f5550102950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 shutdown_connections 2023-12-14T18:20:19.360 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:19.358+0000 7f5556b20700 1 -- 172.21.15.160:0/3794013663 wait complete. 2023-12-14T18:20:19.361 INFO:teuthology.orchestra.run.smithi160.stderr:dumped monmap epoch 3 2023-12-14T18:20:19.952 INFO:tasks.cephadm:Generating final ceph.conf file... 2023-12-14T18:20:19.952 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph config generate-minimal-conf 2023-12-14T18:20:20.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:20.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:20.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:20.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:19 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/3794013663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:20:20.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:20.278 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:20:20.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:20.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:20.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:20.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:19 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/3794013663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:20:20.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:20.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:20.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:20.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:20.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:19 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/3794013663' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2023-12-14T18:20:20.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:20:21.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.205+0000 7fb83fccb700 1 -- 172.21.15.93:0/459466627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb838072e10 msgr2=0x7fb8380731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:21.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.205+0000 7fb83fccb700 1 --2- 172.21.15.93:0/459466627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb838072e10 0x7fb8380731e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fb834009910 tx=0x7fb834009c20 comp rx=0 tx=0).stop 2023-12-14T18:20:21.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.205+0000 7fb83fccb700 1 -- 172.21.15.93:0/459466627 shutdown_connections 2023-12-14T18:20:21.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.205+0000 7fb83fccb700 1 --2- 172.21.15.93:0/459466627 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb838072e10 0x7fb8380731e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:21.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.205+0000 7fb83fccb700 1 -- 172.21.15.93:0/459466627 >> 172.21.15.93:0/459466627 conn(0x7fb8380fb870 msgr2=0x7fb8380fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:21.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.205+0000 7fb83fccb700 1 -- 172.21.15.93:0/459466627 shutdown_connections 2023-12-14T18:20:21.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.205+0000 7fb83fccb700 1 -- 172.21.15.93:0/459466627 wait complete. 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.206+0000 7fb83fccb700 1 Processor -- start 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.206+0000 7fb83fccb700 1 -- start start 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.206+0000 7fb83fccb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb838072e10 0x7fb838197520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.206+0000 7fb83fccb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb838197a60 0x7fb83819bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.206+0000 7fb83fccb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb83819c410 0x7fb83819c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.206+0000 7fb83fccb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb838198130 con 0x7fb83819c410 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.206+0000 7fb83fccb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8381a4180 con 0x7fb838197a60 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.206+0000 7fb83fccb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8381a42c0 con 0x7fb838072e10 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83e268700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb83819c410 0x7fb83819c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83e268700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb83819c410 0x7fb83819c8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59974/0 (socket says 172.21.15.93:59974) 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83e268700 1 -- 172.21.15.93:0/2000922741 learned_addr learned my addr 172.21.15.93:0/2000922741 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83da67700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb838072e10 0x7fb838197520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:21.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83d266700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb838197a60 0x7fb83819bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83e268700 1 -- 172.21.15.93:0/2000922741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb838072e10 msgr2=0x7fb838197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83da67700 1 -- 172.21.15.93:0/2000922741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb838072e10 msgr2=0x7fb838197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83da67700 1 -- 172.21.15.93:0/2000922741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb838072e10 msgr2=0x7fb838197520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83da67700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb838072e10 0x7fb838197520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83d266700 1 -- 172.21.15.93:0/2000922741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb838197a60 msgr2=0x7fb83819bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83d266700 1 -- 172.21.15.93:0/2000922741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb838197a60 msgr2=0x7fb83819bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83d266700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb838197a60 0x7fb83819bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83da67700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb838072e10 0x7fb838197520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83d266700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb838197a60 0x7fb83819bed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83e268700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb838072e10 0x7fb838197520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83e268700 1 -- 172.21.15.93:0/2000922741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb838197a60 msgr2=0x7fb83819bed0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83e268700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb838197a60 0x7fb83819bed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83e268700 1 -- 172.21.15.93:0/2000922741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8340095f0 con 0x7fb83819c410 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83e268700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb83819c410 0x7fb83819c8b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb83000ba80 tx=0x7fb83000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb82effd700 1 -- 172.21.15.93:0/2000922741 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb83000ca50 con 0x7fb83819c410 2023-12-14T18:20:21.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb82effd700 1 -- 172.21.15.93:0/2000922741 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb83000cbb0 con 0x7fb83819c410 2023-12-14T18:20:21.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb83810c320 con 0x7fb83819c410 2023-12-14T18:20:21.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb82effd700 1 -- 172.21.15.93:0/2000922741 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8300075b0 con 0x7fb83819c410 2023-12-14T18:20:21.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.207+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb83810c190 con 0x7fb83819c410 2023-12-14T18:20:21.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.208+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb83810c4b0 con 0x7fb83819c410 2023-12-14T18:20:21.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.209+0000 7fb82effd700 1 -- 172.21.15.93:0/2000922741 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7fb83000cd20 con 0x7fb83819c410 2023-12-14T18:20:21.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.209+0000 7fb82effd700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb824038cf0 0x7fb82403b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:21.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.209+0000 7fb82effd700 1 -- 172.21.15.93:0/2000922741 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7fb83004aff0 con 0x7fb83819c410 2023-12-14T18:20:21.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.209+0000 7fb83da67700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb824038cf0 0x7fb82403b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:21.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.209+0000 7fb83da67700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb824038cf0 0x7fb82403b1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb834006010 tx=0x7fb83400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:21.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.213+0000 7fb82effd700 1 -- 172.21.15.93:0/2000922741 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb830007710 con 0x7fb83819c410 2023-12-14T18:20:21.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:21.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:21.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:21.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:21.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:21.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:21.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:21.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.418+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fb838109990 con 0x7fb83819c410 2023-12-14T18:20:21.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.419+0000 7fb82effd700 1 -- 172.21.15.93:0/2000922741 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+273 (secure 0 0 0) 0x7fb830019030 con 0x7fb83819c410 2023-12-14T18:20:21.420 INFO:teuthology.orchestra.run.smithi093.stdout:# minimal ceph.conf for 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:21.420 INFO:teuthology.orchestra.run.smithi093.stdout:[global] 2023-12-14T18:20:21.420 INFO:teuthology.orchestra.run.smithi093.stdout: fsid = 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:20:21.420 INFO:teuthology.orchestra.run.smithi093.stdout: mon_host = [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] 2023-12-14T18:20:21.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb824038cf0 msgr2=0x7fb82403b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:21.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb824038cf0 0x7fb82403b1a0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb834006010 tx=0x7fb83400b540 comp rx=0 tx=0).stop 2023-12-14T18:20:21.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb83819c410 msgr2=0x7fb83819c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:21.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb83819c410 0x7fb83819c8b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fb83000ba80 tx=0x7fb83000be40 comp rx=0 tx=0).stop 2023-12-14T18:20:21.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 shutdown_connections 2023-12-14T18:20:21.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb824038cf0 0x7fb82403b1a0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:21.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb83819c410 0x7fb83819c8b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:21.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb838072e10 0x7fb838197520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:21.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 --2- 172.21.15.93:0/2000922741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb838197a60 0x7fb83819bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:21.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 >> 172.21.15.93:0/2000922741 conn(0x7fb8380fb870 msgr2=0x7fb838106ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:21.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 shutdown_connections 2023-12-14T18:20:21.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:21.421+0000 7fb83fccb700 1 -- 172.21.15.93:0/2000922741 wait complete. 2023-12-14T18:20:22.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:22.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Updating smithi093:/etc/ceph/ceph.conf 2023-12-14T18:20:22.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Updating smithi097:/etc/ceph/ceph.conf 2023-12-14T18:20:22.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Updating smithi160:/etc/ceph/ceph.conf 2023-12-14T18:20:22.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:22.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:22.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:22.034 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Reconfiguring mon.a (unknown last config time)... 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: Reconfiguring daemon mon.a on smithi093 2023-12-14T18:20:22.035 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:21 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2000922741' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:22.036 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2023-12-14T18:20:22.036 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:20:22.036 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.conf 2023-12-14T18:20:22.066 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:20:22.066 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:20:22.133 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:20:22.133 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.conf 2023-12-14T18:20:22.171 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:20:22.171 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Updating smithi093:/etc/ceph/ceph.conf 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Updating smithi097:/etc/ceph/ceph.conf 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Updating smithi160:/etc/ceph/ceph.conf 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.237 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.238 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.238 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.238 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:22.238 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Reconfiguring mon.a (unknown last config time)... 2023-12-14T18:20:22.238 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:22.238 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:20:22.238 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:22.238 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: Reconfiguring daemon mon.a on smithi093 2023-12-14T18:20:22.238 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:21 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2000922741' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:22.241 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:20:22.241 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.conf 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Updating smithi093:/etc/ceph/ceph.conf 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Updating smithi097:/etc/ceph/ceph.conf 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Updating smithi160:/etc/ceph/ceph.conf 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:22.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Reconfiguring mon.a (unknown last config time)... 2023-12-14T18:20:22.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:22.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:20:22.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:22.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: Reconfiguring daemon mon.a on smithi093 2023-12-14T18:20:22.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:21 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2000922741' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:22.275 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:20:22.275 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:20:22.345 INFO:tasks.cephadm:Adding mgr.x on smithi093 2023-12-14T18:20:22.345 INFO:tasks.cephadm:Adding mgr.y on smithi097 2023-12-14T18:20:22.346 INFO:tasks.cephadm:Adding mgr.z on smithi160 2023-12-14T18:20:22.346 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch apply mgr '3;smithi093=x;smithi097=y;smithi160=z' 2023-12-14T18:20:22.686 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:20:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: Reconfiguring mon.b (monmap changed)... 2023-12-14T18:20:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:20:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: Reconfiguring daemon mon.b on smithi097 2023-12-14T18:20:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: Reconfiguring mon.b (monmap changed)... 2023-12-14T18:20:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:20:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: Reconfiguring daemon mon.b on smithi097 2023-12-14T18:20:23.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: Reconfiguring mon.b (monmap changed)... 2023-12-14T18:20:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:20:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: Reconfiguring daemon mon.b on smithi097 2023-12-14T18:20:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:23.737 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/4067449124 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 msgr2=0x7f9adc100250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:23.737 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 --2- 172.21.15.160:0/4067449124 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 0x7f9adc100250 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f9acc009910 tx=0x7f9acc009c20 comp rx=0 tx=0).stop 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/4067449124 shutdown_connections 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 --2- 172.21.15.160:0/4067449124 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 0x7f9adc100250 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/4067449124 >> 172.21.15.160:0/4067449124 conn(0x7f9adc0fb3f0 msgr2=0x7f9adc0fd820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/4067449124 shutdown_connections 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/4067449124 wait complete. 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 Processor -- start 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 -- start start 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.733+0000 7f9ae3d6d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 0x7f9adc195050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae3d6d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9adc195590 0x7f9adc19a600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:23.738 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae3d6d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9adc195a00 0x7f9adc19cb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae3d6d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19d310 con 0x7f9adc0ffe40 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae3d6d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19d480 con 0x7f9adc195a00 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae3d6d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9adc19d640 con 0x7f9adc195590 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1b09700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 0x7f9adc195050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1b09700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 0x7f9adc195050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.160:60016/0 (socket says 172.21.15.160:60016) 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1b09700 1 -- 172.21.15.160:0/1685048374 learned_addr learned my addr 172.21.15.160:0/1685048374 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae230a700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9adc195a00 0x7f9adc19cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1308700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9adc195590 0x7f9adc19a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1b09700 1 -- 172.21.15.160:0/1685048374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9adc195590 msgr2=0x7f9adc19a600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:23.739 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1b09700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9adc195590 0x7f9adc19a600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1b09700 1 -- 172.21.15.160:0/1685048374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9adc195a00 msgr2=0x7f9adc19cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1b09700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9adc195a00 0x7f9adc19cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1b09700 1 -- 172.21.15.160:0/1685048374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9acc0095f0 con 0x7f9adc0ffe40 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae230a700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9adc195a00 0x7f9adc19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1308700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9adc195590 0x7f9adc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae1b09700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 0x7f9adc195050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9acc004690 tx=0x7f9acc004d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ad2ffd700 1 -- 172.21.15.160:0/1685048374 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc01d070 con 0x7f9adc0ffe40 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9adc105c40 con 0x7f9adc0ffe40 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ad2ffd700 1 -- 172.21.15.160:0/1685048374 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9acc00bea0 con 0x7f9adc0ffe40 2023-12-14T18:20:23.740 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ad2ffd700 1 -- 172.21.15.160:0/1685048374 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9acc00f460 con 0x7f9adc0ffe40 2023-12-14T18:20:23.741 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.734+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9adc105ab0 con 0x7f9adc0ffe40 2023-12-14T18:20:23.741 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.735+0000 7f9ad2ffd700 1 -- 172.21.15.160:0/1685048374 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f9acc022470 con 0x7f9adc0ffe40 2023-12-14T18:20:23.741 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.735+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9adc18e970 con 0x7f9adc0ffe40 2023-12-14T18:20:23.741 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.735+0000 7f9ad2ffd700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9ac8038cf0 0x7f9ac803b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:23.741 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.736+0000 7f9ad2ffd700 1 -- 172.21.15.160:0/1685048374 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f9acc04c7e0 con 0x7f9adc0ffe40 2023-12-14T18:20:23.741 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.736+0000 7f9ae1308700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9ac8038cf0 0x7f9ac803b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:23.741 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.736+0000 7f9ae1308700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9ac8038cf0 0x7f9ac803b1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80079b0 tx=0x7f9ad8008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:23.742 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.741+0000 7f9ad2ffd700 1 -- 172.21.15.160:0/1685048374 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9acc022770 con 0x7f9adc0ffe40 2023-12-14T18:20:23.952 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.951+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi097=y;smithi160=z", "target": ["mon-mgr", ""]}) v1 -- 0x7f9adc1008d0 con 0x7f9ac8038cf0 2023-12-14T18:20:23.958 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.957+0000 7f9ad2ffd700 1 -- 172.21.15.160:0/1685048374 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f9adc1008d0 con 0x7f9ac8038cf0 2023-12-14T18:20:23.959 INFO:teuthology.orchestra.run.smithi160.stdout:Scheduled mgr update... 2023-12-14T18:20:23.961 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9ac8038cf0 msgr2=0x7f9ac803b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:23.961 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9ac8038cf0 0x7f9ac803b1a0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9ad80079b0 tx=0x7f9ad8008040 comp rx=0 tx=0).stop 2023-12-14T18:20:23.961 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 msgr2=0x7f9adc195050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:23.962 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 0x7f9adc195050 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f9acc004690 tx=0x7f9acc004d60 comp rx=0 tx=0).stop 2023-12-14T18:20:23.962 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 shutdown_connections 2023-12-14T18:20:23.962 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9ac8038cf0 0x7f9ac803b1a0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:23.962 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9adc0ffe40 0x7f9adc195050 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:23.962 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9adc195a00 0x7f9adc19cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:23.962 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 --2- 172.21.15.160:0/1685048374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9adc195590 0x7f9adc19a600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:23.962 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.959+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 >> 172.21.15.160:0/1685048374 conn(0x7f9adc0fb3f0 msgr2=0x7f9adc0fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:23.962 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.960+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 shutdown_connections 2023-12-14T18:20:23.962 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:20:23.960+0000 7f9ae3d6d700 1 -- 172.21.15.160:0/1685048374 wait complete. 2023-12-14T18:20:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:24 smithi160 ceph-mon[43840]: Reconfiguring mon.c (monmap changed)... 2023-12-14T18:20:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:24 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:24 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:20:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:24 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:24 smithi160 ceph-mon[43840]: Reconfiguring daemon mon.c on smithi160 2023-12-14T18:20:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:24 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:24.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:24 smithi093 ceph-mon[42070]: Reconfiguring mon.c (monmap changed)... 2023-12-14T18:20:24.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:24 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:24.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:24 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:20:24.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:24 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:24.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:24 smithi093 ceph-mon[42070]: Reconfiguring daemon mon.c on smithi160 2023-12-14T18:20:24.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:24 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:24.581 DEBUG:teuthology.orchestra.run.smithi097:mgr.y> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.y.service 2023-12-14T18:20:24.583 DEBUG:teuthology.orchestra.run.smithi160:mgr.z> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.z.service 2023-12-14T18:20:24.586 INFO:tasks.cephadm:Deploying OSDs... 2023-12-14T18:20:24.586 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:20:24.586 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/scratch_devs of=/dev/stdout 2023-12-14T18:20:24.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:24 smithi097 ceph-mon[43486]: Reconfiguring mon.c (monmap changed)... 2023-12-14T18:20:24.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:24 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2023-12-14T18:20:24.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:24 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2023-12-14T18:20:24.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:24 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:24.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:24 smithi097 ceph-mon[43486]: Reconfiguring daemon mon.c on smithi160 2023-12-14T18:20:24.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:24 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:24.608 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-14T18:20:24.608 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_1 2023-12-14T18:20:24.616 INFO:journalctl@ceph.mgr.z.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:20:24.617 INFO:journalctl@ceph.mgr.y.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:20:24.668 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-14T18:20:24.668 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:24.668 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 119438 Links: 1 2023-12-14T18:20:24.668 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:24.668 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:24.668 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-12-14 18:19:44.766535510 +0000 2023-12-14T18:20:24.668 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-12-14 18:12:30.719169547 +0000 2023-12-14T18:20:24.669 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-12-14 18:12:30.719169547 +0000 2023-12-14T18:20:24.669 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-12-14T18:20:24.669 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-14T18:20:24.737 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-12-14T18:20:24.737 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-12-14T18:20:24.737 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000239645 s, 2.1 MB/s 2023-12-14T18:20:24.738 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-14T18:20:24.798 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_2 2023-12-14T18:20:24.856 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-14T18:20:24.856 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:24.856 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 119518 Links: 1 2023-12-14T18:20:24.856 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:24.856 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:24.856 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-12-14 18:19:44.766535510 +0000 2023-12-14T18:20:24.856 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-12-14 18:12:31.231158130 +0000 2023-12-14T18:20:24.856 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-12-14 18:12:31.231158130 +0000 2023-12-14T18:20:24.856 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-12-14T18:20:24.856 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-14T18:20:24.923 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-12-14T18:20:24.923 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-12-14T18:20:24.923 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000227025 s, 2.3 MB/s 2023-12-14T18:20:24.924 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-14T18:20:24.982 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_3 2023-12-14T18:20:25.040 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-14T18:20:25.040 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:25.040 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 119614 Links: 1 2023-12-14T18:20:25.040 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:25.040 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:25.040 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-12-14 18:19:44.766535510 +0000 2023-12-14T18:20:25.040 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-12-14 18:12:31.748146602 +0000 2023-12-14T18:20:25.040 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-12-14 18:12:31.748146602 +0000 2023-12-14T18:20:25.040 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-12-14T18:20:25.040 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-14T18:20:25.105 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-12-14T18:20:25.106 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-12-14T18:20:25.106 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000242258 s, 2.1 MB/s 2023-12-14T18:20:25.106 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-14T18:20:25.163 DEBUG:teuthology.orchestra.run.smithi093:> stat /dev/vg_nvme/lv_4 2023-12-14T18:20:25.220 INFO:teuthology.orchestra.run.smithi093.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-14T18:20:25.220 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:25.220 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 6h/6d Inode: 119692 Links: 1 2023-12-14T18:20:25.220 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:25.220 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:25.221 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-12-14 18:19:44.766535510 +0000 2023-12-14T18:20:25.221 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-12-14 18:12:32.253135341 +0000 2023-12-14T18:20:25.221 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-12-14 18:12:32.253135341 +0000 2023-12-14T18:20:25.221 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: - 2023-12-14T18:20:25.221 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-14T18:20:25.286 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records in 2023-12-14T18:20:25.286 INFO:teuthology.orchestra.run.smithi093.stderr:1+0 records out 2023-12-14T18:20:25.286 INFO:teuthology.orchestra.run.smithi093.stderr:512 bytes copied, 0.000240182 s, 2.1 MB/s 2023-12-14T18:20:25.287 DEBUG:teuthology.orchestra.run.smithi093:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-14T18:20:25.345 DEBUG:teuthology.orchestra.run.smithi097:> set -ex 2023-12-14T18:20:25.345 DEBUG:teuthology.orchestra.run.smithi097:> dd if=/scratch_devs of=/dev/stdout 2023-12-14T18:20:25.365 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-14T18:20:25.366 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_1 2023-12-14T18:20:25.417 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:25.417 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi097=y;smithi160=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:25.417 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: Saving service mgr spec with placement smithi093=x;smithi097=y;smithi160=z;count:3 2023-12-14T18:20:25.417 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:25.417 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:25.417 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:25.417 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:25.417 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:25.418 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:25.418 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:20:25.418 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:20:25.418 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:20:25.418 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:25.426 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-14T18:20:25.426 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:25.426 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 124377 Links: 1 2023-12-14T18:20:25.426 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:25.426 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:25.426 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-12-14 18:20:13.402922318 +0000 2023-12-14T18:20:25.426 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-12-14 18:13:30.227272734 +0000 2023-12-14T18:20:25.427 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-12-14 18:13:30.227272734 +0000 2023-12-14T18:20:25.427 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-12-14T18:20:25.427 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-14T18:20:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-12-14T18:20:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-12-14T18:20:25.499 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000256078 s, 2.0 MB/s 2023-12-14T18:20:25.500 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-14T18:20:25.561 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_2 2023-12-14T18:20:25.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:25.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi097=y;smithi160=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:25.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: Saving service mgr spec with placement smithi093=x;smithi097=y;smithi160=z;count:3 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:20:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:25.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:25.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='client.14211 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi093=x;smithi097=y;smithi160=z", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: Saving service mgr spec with placement smithi093=x;smithi097=y;smithi160=z;count:3 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:20:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:25.612 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-14T18:20:25.612 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:25.612 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 123162 Links: 1 2023-12-14T18:20:25.612 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:25.612 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:25.612 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-12-14 18:20:13.402922318 +0000 2023-12-14T18:20:25.612 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-12-14 18:13:30.725264133 +0000 2023-12-14T18:20:25.612 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-12-14 18:13:30.725264133 +0000 2023-12-14T18:20:25.612 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-12-14T18:20:25.612 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-14T18:20:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-12-14T18:20:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-12-14T18:20:25.681 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000232094 s, 2.2 MB/s 2023-12-14T18:20:25.682 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-14T18:20:25.741 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_3 2023-12-14T18:20:25.799 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-14T18:20:25.799 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:25.799 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 129251 Links: 1 2023-12-14T18:20:25.800 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:25.800 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:25.800 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-12-14 18:20:13.402922318 +0000 2023-12-14T18:20:25.800 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-12-14 18:13:31.240255238 +0000 2023-12-14T18:20:25.800 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-12-14 18:13:31.240255238 +0000 2023-12-14T18:20:25.800 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-12-14T18:20:25.800 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-14T18:20:25.865 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-12-14T18:20:25.866 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-12-14T18:20:25.866 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000246799 s, 2.1 MB/s 2023-12-14T18:20:25.867 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-14T18:20:25.925 DEBUG:teuthology.orchestra.run.smithi097:> stat /dev/vg_nvme/lv_4 2023-12-14T18:20:25.983 INFO:teuthology.orchestra.run.smithi097.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-14T18:20:25.983 INFO:teuthology.orchestra.run.smithi097.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:25.983 INFO:teuthology.orchestra.run.smithi097.stdout:Device: 6h/6d Inode: 131087 Links: 1 2023-12-14T18:20:25.983 INFO:teuthology.orchestra.run.smithi097.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:25.983 INFO:teuthology.orchestra.run.smithi097.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:25.983 INFO:teuthology.orchestra.run.smithi097.stdout:Access: 2023-12-14 18:20:13.403922301 +0000 2023-12-14T18:20:25.983 INFO:teuthology.orchestra.run.smithi097.stdout:Modify: 2023-12-14 18:13:31.749246446 +0000 2023-12-14T18:20:25.983 INFO:teuthology.orchestra.run.smithi097.stdout:Change: 2023-12-14 18:13:31.749246446 +0000 2023-12-14T18:20:25.983 INFO:teuthology.orchestra.run.smithi097.stdout: Birth: - 2023-12-14T18:20:25.984 DEBUG:teuthology.orchestra.run.smithi097:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-14T18:20:25.996 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:25 smithi160 systemd[1]: Starting Ceph mgr.z for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:20:26.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records in 2023-12-14T18:20:26.049 INFO:teuthology.orchestra.run.smithi097.stderr:1+0 records out 2023-12-14T18:20:26.049 INFO:teuthology.orchestra.run.smithi097.stderr:512 bytes copied, 0.000216064 s, 2.4 MB/s 2023-12-14T18:20:26.050 DEBUG:teuthology.orchestra.run.smithi097:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-14T18:20:26.108 DEBUG:teuthology.orchestra.run.smithi160:> set -ex 2023-12-14T18:20:26.108 DEBUG:teuthology.orchestra.run.smithi160:> dd if=/scratch_devs of=/dev/stdout 2023-12-14T18:20:26.126 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-14T18:20:26.126 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_1 2023-12-14T18:20:26.186 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-14T18:20:26.186 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:26.186 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 115527 Links: 1 2023-12-14T18:20:26.186 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:26.186 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:26.186 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-12-14 18:20:25.777204678 +0000 2023-12-14T18:20:26.186 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-12-14 18:12:41.979265192 +0000 2023-12-14T18:20:26.186 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-12-14 18:12:41.979265192 +0000 2023-12-14T18:20:26.186 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2023-12-14T18:20:26.187 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-14T18:20:26.256 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-12-14T18:20:26.256 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-12-14T18:20:26.257 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000252442 s, 2.0 MB/s 2023-12-14T18:20:26.258 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-14T18:20:26.316 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_2 2023-12-14T18:20:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:20:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: Deploying daemon mgr.z on smithi160 2023-12-14T18:20:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:20:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:20:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:20:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:26.373 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-14T18:20:26.373 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:26.373 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 127076 Links: 1 2023-12-14T18:20:26.373 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:26.373 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:26.373 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-12-14 18:20:25.777204678 +0000 2023-12-14T18:20:26.373 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-12-14 18:12:42.491257466 +0000 2023-12-14T18:20:26.374 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-12-14 18:12:42.491257466 +0000 2023-12-14T18:20:26.374 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2023-12-14T18:20:26.374 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-14T18:20:26.437 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:26 smithi160 bash[45657]: 9a37f4fb58a61c918b088d0e66100a313687bdf79d089583b46b7f6c8c0bed6e 2023-12-14T18:20:26.437 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:26 smithi160 systemd[1]: Started Ceph mgr.z for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:20:26.437 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mgr[45713]: -- 172.21.15.160:0/3479945212 <== mon.1 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562be03a7520 con 0x562be037cc00 2023-12-14T18:20:26.437 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:26.437 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:20:26.438 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: Deploying daemon mgr.z on smithi160 2023-12-14T18:20:26.438 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:26.438 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:26.438 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:26.438 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:20:26.438 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:20:26.438 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:20:26.438 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:26.439 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-12-14T18:20:26.439 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-12-14T18:20:26.439 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000216897 s, 2.4 MB/s 2023-12-14T18:20:26.441 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-14T18:20:26.498 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_3 2023-12-14T18:20:26.556 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-14T18:20:26.556 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:26.556 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 125212 Links: 1 2023-12-14T18:20:26.556 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:26.556 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:26.557 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-12-14 18:20:25.777204678 +0000 2023-12-14T18:20:26.557 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-12-14 18:12:42.989249951 +0000 2023-12-14T18:20:26.557 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-12-14 18:12:42.989249951 +0000 2023-12-14T18:20:26.557 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2023-12-14T18:20:26.557 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-14T18:20:26.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:26.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:20:26.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: Deploying daemon mgr.z on smithi160 2023-12-14T18:20:26.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:26.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:26.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:26.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:20:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2023-12-14T18:20:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:20:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:26.622 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-12-14T18:20:26.622 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-12-14T18:20:26.622 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000213006 s, 2.4 MB/s 2023-12-14T18:20:26.623 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-14T18:20:26.680 DEBUG:teuthology.orchestra.run.smithi160:> stat /dev/vg_nvme/lv_4 2023-12-14T18:20:26.733 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:26.532+0000 7f6e03a48200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:20:26.733 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:26.640+0000 7f6e03a48200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:20:26.738 INFO:teuthology.orchestra.run.smithi160.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-14T18:20:26.738 INFO:teuthology.orchestra.run.smithi160.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-14T18:20:26.738 INFO:teuthology.orchestra.run.smithi160.stdout:Device: 6h/6d Inode: 128091 Links: 1 2023-12-14T18:20:26.739 INFO:teuthology.orchestra.run.smithi160.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:20:26.739 INFO:teuthology.orchestra.run.smithi160.stdout:Context: system_u:object_r:device_t:s0 2023-12-14T18:20:26.739 INFO:teuthology.orchestra.run.smithi160.stdout:Access: 2023-12-14 18:20:25.777204678 +0000 2023-12-14T18:20:26.739 INFO:teuthology.orchestra.run.smithi160.stdout:Modify: 2023-12-14 18:12:43.518241968 +0000 2023-12-14T18:20:26.739 INFO:teuthology.orchestra.run.smithi160.stdout:Change: 2023-12-14 18:12:43.518241968 +0000 2023-12-14T18:20:26.739 INFO:teuthology.orchestra.run.smithi160.stdout: Birth: - 2023-12-14T18:20:26.739 DEBUG:teuthology.orchestra.run.smithi160:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-14T18:20:26.804 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records in 2023-12-14T18:20:26.805 INFO:teuthology.orchestra.run.smithi160.stderr:1+0 records out 2023-12-14T18:20:26.805 INFO:teuthology.orchestra.run.smithi160.stderr:512 bytes copied, 0.000258411 s, 2.0 MB/s 2023-12-14T18:20:26.806 DEBUG:teuthology.orchestra.run.smithi160:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-14T18:20:26.864 INFO:tasks.cephadm:Deploying osd.0 on smithi093 with /dev/vg_nvme/lv_4... 2023-12-14T18:20:26.864 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-14T18:20:26.996 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:26.732+0000 7f6e03a48200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:20:26.996 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:26.820+0000 7f6e03a48200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:20:27.182 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:20:27.495 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:27 smithi097 ceph-mon[43486]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:27.495 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:27 smithi097 ceph-mon[43486]: Deploying daemon mgr.y on smithi097 2023-12-14T18:20:27.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:27 smithi093 ceph-mon[42070]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:27 smithi093 ceph-mon[42070]: Deploying daemon mgr.y on smithi097 2023-12-14T18:20:27.683 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:27.447+0000 7f6e03a48200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:27.683 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:27 smithi160 ceph-mon[43840]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:27.683 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:27 smithi160 ceph-mon[43840]: Deploying daemon mgr.y on smithi097 2023-12-14T18:20:27.842 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:27 smithi097 systemd[1]: Starting Ceph mgr.y for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:20:27.969 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:27.682+0000 7f6e03a48200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:20:27.969 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:27.768+0000 7f6e03a48200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:20:27.969 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:27.877+0000 7f6e03a48200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:27.969 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:27.968+0000 7f6e03a48200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:20:28.246 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:28.167+0000 7f6e03a48200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:20:28.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:28 smithi097 ceph-mgr[44957]: -- 172.21.15.97:0/1216782666 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5569eb3fb520 con 0x5569eb3d0c00 2023-12-14T18:20:28.343 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:28 smithi097 bash[44919]: a97c760b82c02adad150712cacf3811dac70ecdc12cb9907c13239b0ae132152 2023-12-14T18:20:28.343 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:28 smithi097 systemd[1]: Started Ceph mgr.y for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:20:28.461 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-14T18:20:28.461 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-14T18:20:28.462 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-12-14T18:20:28.462 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-12-14T18:20:28.462 INFO:teuthology.orchestra.run.smithi093.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0111875 s, 937 MB/s 2023-12-14T18:20:28.462 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-12-14T18:20:28.641 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:28.341+0000 7f8ac2fe8200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:20:28.642 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:28.450+0000 7f8ac2fe8200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:20:28.642 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:28.545+0000 7f8ac2fe8200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:20:28.642 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:28.639+0000 7f8ac2fe8200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:20:28.674 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:28.255+0000 7f6e03a48200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:20:28.826 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi093:vg_nvme/lv_4 2023-12-14T18:20:28.960 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:28.673+0000 7f6e03a48200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:20:29.136 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:20:29.246 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:28.959+0000 7f6e03a48200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:20:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:29 smithi160 ceph-mon[43840]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:29 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:29 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:29 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:29 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:29 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:29.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:29 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:29.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:29.284+0000 7f8ac2fe8200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-mon[43486]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:29.421 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:29 smithi093 ceph-mon[42070]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:29.421 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:29 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.421 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:29 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.421 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:29 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.421 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:29 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:29.421 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:29 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:29.422 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:29 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:29.746 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:29 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:29.265+0000 7f6e03a48200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:20:29.746 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:29 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:29.364+0000 7f6e03a48200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:20:29.802 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:29.518+0000 7f8ac2fe8200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:20:29.802 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:29.603+0000 7f8ac2fe8200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:20:29.802 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:29.709+0000 7f8ac2fe8200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:30.079 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:29.800+0000 7f8ac2fe8200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:20:30.079 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:29.993+0000 7f8ac2fe8200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.201+0000 7f0762cf5700 1 -- 172.21.15.93:0/2870946900 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c0ff1e0 msgr2=0x7f075c0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.201+0000 7f0762cf5700 1 --2- 172.21.15.93:0/2870946900 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c0ff1e0 0x7f075c0ff5f0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f075400b0a0 tx=0x7f075400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.201+0000 7f0762cf5700 1 -- 172.21.15.93:0/2870946900 shutdown_connections 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.201+0000 7f0762cf5700 1 --2- 172.21.15.93:0/2870946900 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c0ff1e0 0x7f075c0ff5f0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.201+0000 7f0762cf5700 1 --2- 172.21.15.93:0/2870946900 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c100d70 0x7f075c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.201+0000 7f0762cf5700 1 --2- 172.21.15.93:0/2870946900 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f075c1003e0 0x7f075c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.201+0000 7f0762cf5700 1 -- 172.21.15.93:0/2870946900 >> 172.21.15.93:0/2870946900 conn(0x7f075c0fa7f0 msgr2=0x7f075c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.201+0000 7f0762cf5700 1 -- 172.21.15.93:0/2870946900 shutdown_connections 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.201+0000 7f0762cf5700 1 -- 172.21.15.93:0/2870946900 wait complete. 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0762cf5700 1 Processor -- start 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0762cf5700 1 -- start start 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0762cf5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c0ff1e0 0x7f075c1961d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0762cf5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c1003e0 0x7f075c196710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:30.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0762cf5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f075c100d70 0x7f075c19d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0762cf5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c19dd60 con 0x7f075c1003e0 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0762cf5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c19dea0 con 0x7f075c0ff1e0 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0762cf5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f075c19e030 con 0x7f075c100d70 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f075bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c1003e0 0x7f075c196710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f075bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c1003e0 0x7f075c196710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45200/0 (socket says 172.21.15.93:45200) 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f075bfff700 1 -- 172.21.15.93:0/241626010 learned_addr learned my addr 172.21.15.93:0/241626010 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0760a91700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c0ff1e0 0x7f075c1961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0761292700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f075c100d70 0x7f075c19d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f075bfff700 1 -- 172.21.15.93:0/241626010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f075c100d70 msgr2=0x7f075c19d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f075bfff700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f075c100d70 0x7f075c19d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f075bfff700 1 -- 172.21.15.93:0/241626010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c0ff1e0 msgr2=0x7f075c1961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f075bfff700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c0ff1e0 0x7f075c1961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f075bfff700 1 -- 172.21.15.93:0/241626010 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0754009d00 con 0x7f075c1003e0 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0761292700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f075c100d70 0x7f075c19d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f075bfff700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c1003e0 0x7f075c196710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f075000d980 tx=0x7f075000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:30.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0759ffb700 1 -- 172.21.15.93:0/241626010 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0750009b40 con 0x7f075c1003e0 2023-12-14T18:20:30.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.202+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f075c10ab30 con 0x7f075c1003e0 2023-12-14T18:20:30.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.203+0000 7f0759ffb700 1 -- 172.21.15.93:0/241626010 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f075000b6e0 con 0x7f075c1003e0 2023-12-14T18:20:30.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.203+0000 7f0759ffb700 1 -- 172.21.15.93:0/241626010 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f075000f620 con 0x7f075c1003e0 2023-12-14T18:20:30.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.203+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f075c10a9a0 con 0x7f075c1003e0 2023-12-14T18:20:30.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.203+0000 7f0759ffb700 1 -- 172.21.15.93:0/241626010 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 13) v1 ==== 45539+0+0 (secure 0 0 0) 0x7f075000f780 con 0x7f075c1003e0 2023-12-14T18:20:30.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.204+0000 7f0759ffb700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f07440389e0 0x7f074403ae90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:30.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.204+0000 7f0759ffb700 1 -- 172.21.15.93:0/241626010 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1155+0+0 (secure 0 0 0) 0x7f075004b660 con 0x7f075c1003e0 2023-12-14T18:20:30.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.204+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f075c066ea0 con 0x7f075c1003e0 2023-12-14T18:20:30.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.204+0000 7f0760a91700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f07440389e0 0x7f074403ae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:30.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.204+0000 7f0760a91700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f07440389e0 0x7f074403ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0754009fd0 tx=0x7f075400b7b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:30.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.208+0000 7f0759ffb700 1 -- 172.21.15.93:0/241626010 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f075000eb00 con 0x7f075c1003e0 2023-12-14T18:20:30.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:20:30.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:30.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:30.077+0000 7f8ac2fe8200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:20:30.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:20:30.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:30.392 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:30.146+0000 7f6e03a48200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:20:30.392 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:30.243+0000 7f6e03a48200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2023-12-14T18:20:30.393 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2023-12-14T18:20:30.394 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:30.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:30.412+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f075c068ce0 con 0x7f07440389e0 2023-12-14T18:20:30.717 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:30.450+0000 7f8ac2fe8200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:20:30.745 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:30 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:30.391+0000 7f6e03a48200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:20:31.000 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:30 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:30.716+0000 7f8ac2fe8200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:20:31.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:30.998+0000 7f8ac2fe8200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:20:31.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:31.091+0000 7f8ac2fe8200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:20:31.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: Reconfiguring mgr.x (unknown last config time)... 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: Reconfiguring daemon mgr.x on smithi093 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.413 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:31.031+0000 7f6e03a48200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:31.216+0000 7f6e03a48200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: Reconfiguring mgr.x (unknown last config time)... 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: Reconfiguring daemon mgr.x on smithi093 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:31.414 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.415 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.513 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: Reconfiguring mgr.x (unknown last config time)... 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: Reconfiguring daemon mgr.x on smithi093 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.514 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:31.745 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:31.412+0000 7f6e03a48200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:20:32.092 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:31.835+0000 7f8ac2fe8200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:20:32.092 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:31 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:31.930+0000 7f8ac2fe8200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:20:32.092 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:32 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:32.076+0000 7f8ac2fe8200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:20:32.496 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:32.126+0000 7f6e03a48200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:20:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:32 smithi160 ceph-mon[43840]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:32 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:32 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:20:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:32 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:32 smithi093 ceph-mon[42070]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:32 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:32 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:20:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:32 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:32.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:32 smithi097 ceph-mon[43486]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:32.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:32 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:32.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:32 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:20:32.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:32 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:32.897 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:32.612+0000 7f6e03a48200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:32.898 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:32.703+0000 7f6e03a48200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:33.070 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:32 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:32.696+0000 7f8ac2fe8200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:20:33.070 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:32 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:32.874+0000 7f8ac2fe8200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:20:33.239 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:33.238+0000 7f0759ffb700 1 -- 172.21.15.93:0/241626010 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 14) v1 ==== 90599+0+0 (secure 0 0 0) 0x7f075000fb20 con 0x7f075c1003e0 2023-12-14T18:20:33.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:33.068+0000 7f8ac2fe8200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3458151578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4bece78-53dc-4e27-bb2b-32a16baa8936"}]: dispatch 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3458151578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4bece78-53dc-4e27-bb2b-32a16baa8936"}]': finished 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: osdmap e5: 1 total, 0 up, 1 in 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2043501412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: Standby manager daemon z started 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:20:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:20:33.496 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:20:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:20:32.896+0000 7f6e03a48200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:20:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3458151578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4bece78-53dc-4e27-bb2b-32a16baa8936"}]: dispatch 2023-12-14T18:20:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3458151578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4bece78-53dc-4e27-bb2b-32a16baa8936"}]': finished 2023-12-14T18:20:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: osdmap e5: 1 total, 0 up, 1 in 2023-12-14T18:20:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2043501412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:20:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: Standby manager daemon z started 2023-12-14T18:20:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:20:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:20:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:20:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:33 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:20:33.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3458151578' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4bece78-53dc-4e27-bb2b-32a16baa8936"}]: dispatch 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3458151578' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4bece78-53dc-4e27-bb2b-32a16baa8936"}]': finished 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: osdmap e5: 1 total, 0 up, 1 in 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2043501412' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: Standby manager daemon z started 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:20:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:33 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.160:0/1241825342' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:20:34.092 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:33 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:33.771+0000 7f8ac2fe8200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:20:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:34 smithi160 ceph-mon[43840]: mgrmap e14: x(active, since 57s), standbys: z 2023-12-14T18:20:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:34 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:20:34.519 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:34 smithi097 ceph-mon[43486]: mgrmap e14: x(active, since 57s), standbys: z 2023-12-14T18:20:34.519 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:34 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:20:34.519 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:34 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:34.238+0000 7f8ac2fe8200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:20:34.519 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:34 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:34.327+0000 7f8ac2fe8200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:20:34.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:34 smithi093 ceph-mon[42070]: mgrmap e14: x(active, since 57s), standbys: z 2023-12-14T18:20:34.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:34 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:20:34.842 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:20:34 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:20:34.516+0000 7f8ac2fe8200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:20:35.250 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:35.248+0000 7f0759ffb700 1 -- 172.21.15.93:0/241626010 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f0750019c00 con 0x7f075c1003e0 2023-12-14T18:20:35.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:35 smithi093 ceph-mon[42070]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:35.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:35 smithi093 ceph-mon[42070]: Standby manager daemon y started 2023-12-14T18:20:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:35 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:20:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:35 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:20:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:35 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:20:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:35 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:20:35.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:35 smithi097 ceph-mon[43486]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:35 smithi097 ceph-mon[43486]: Standby manager daemon y started 2023-12-14T18:20:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:35 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:20:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:35 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:20:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:35 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:20:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:35 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:20:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:35 smithi160 ceph-mon[43840]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:35 smithi160 ceph-mon[43840]: Standby manager daemon y started 2023-12-14T18:20:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:35 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:20:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:35 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:20:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:35 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:20:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:35 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.97:0/3926047401' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:20:36.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:36 smithi093 ceph-mon[42070]: mgrmap e15: x(active, since 59s), standbys: z, y 2023-12-14T18:20:36.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:20:36.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:36 smithi097 ceph-mon[43486]: mgrmap e15: x(active, since 59s), standbys: z, y 2023-12-14T18:20:36.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:20:36.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:36 smithi160 ceph-mon[43840]: mgrmap e15: x(active, since 59s), standbys: z, y 2023-12-14T18:20:36.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:20:37.437 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:37 smithi093 ceph-mon[42070]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:37.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:37 smithi097 ceph-mon[43486]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:37.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:37 smithi160 ceph-mon[43840]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:38.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:38 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-14T18:20:38.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:38 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:38.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:38 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-14T18:20:38.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:38 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:38.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:38 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2023-12-14T18:20:38.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:38 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:39.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:39 smithi093 ceph-mon[42070]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:39.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:39 smithi093 ceph-mon[42070]: Deploying daemon osd.0 on smithi093 2023-12-14T18:20:39.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:39 smithi097 ceph-mon[43486]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:39.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:39 smithi097 ceph-mon[43486]: Deploying daemon osd.0 on smithi093 2023-12-14T18:20:39.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:39 smithi160 ceph-mon[43840]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:39.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:39 smithi160 ceph-mon[43840]: Deploying daemon osd.0 on smithi093 2023-12-14T18:20:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:41 smithi093 ceph-mon[42070]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:41.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:41 smithi097 ceph-mon[43486]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:41.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:41 smithi160 ceph-mon[43840]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:43.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:42 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:43.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:42 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:43.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:42 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:43.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:42 smithi093 ceph-mon[42070]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:43.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:42 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:43.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:42 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:43.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:42 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:43.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:42 smithi097 ceph-mon[43486]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:43.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:42 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:43.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:42 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:43.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:42 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:43.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:42 smithi160 ceph-mon[43840]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:44.112 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 0 on host 'smithi093' 2023-12-14T18:20:44.113 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.111+0000 7f0759ffb700 1 -- 172.21.15.93:0/241626010 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f075c068ce0 con 0x7f07440389e0 2023-12-14T18:20:44.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f07440389e0 msgr2=0x7f074403ae90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:44.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f07440389e0 0x7f074403ae90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0754009fd0 tx=0x7f075400b7b0 comp rx=0 tx=0).stop 2023-12-14T18:20:44.114 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c1003e0 msgr2=0x7f075c196710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:44.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c1003e0 0x7f075c196710 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f075000d980 tx=0x7f075000dc90 comp rx=0 tx=0).stop 2023-12-14T18:20:44.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 shutdown_connections 2023-12-14T18:20:44.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f07440389e0 0x7f074403ae90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:44.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f075c100d70 0x7f075c19d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:44.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f075c0ff1e0 0x7f075c1961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:44.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 --2- 172.21.15.93:0/241626010 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f075c1003e0 0x7f075c196710 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:44.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 >> 172.21.15.93:0/241626010 conn(0x7f075c0fa7f0 msgr2=0x7f075c0685d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:44.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.113+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 shutdown_connections 2023-12-14T18:20:44.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:20:44.114+0000 7f0762cf5700 1 -- 172.21.15.93:0/241626010 wait complete. 2023-12-14T18:20:44.522 DEBUG:teuthology.orchestra.run.smithi093:osd.0> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.0.service 2023-12-14T18:20:44.524 INFO:tasks.cephadm:Deploying osd.1 on smithi097 with /dev/vg_nvme/lv_4... 2023-12-14T18:20:44.525 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-14T18:20:44.553 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.554 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.555 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:44.558 INFO:journalctl@ceph.osd.0.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:20:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:20:44.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:44.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:44.838 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:20:44.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:44.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:20:46.044 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:20:45 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-0[54878]: 2023-12-14T18:20:45.774+0000 7f08c17de640 -1 osd.0 0 log_to_monitors true 2023-12-14T18:20:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-14T18:20:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-14T18:20:46.123 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-12-14T18:20:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-12-14T18:20:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0150611 s, 696 MB/s 2023-12-14T18:20:46.123 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-12-14T18:20:46.467 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_4 2023-12-14T18:20:46.770 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:20:46.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-14T18:20:46.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:46.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi093 2023-12-14T18:20:46.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:46.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:46.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:20:46.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi093 to 17113M 2023-12-14T18:20:46.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:46.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:46.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:46.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:46.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:46.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:20:46.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:46 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:20:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-14T18:20:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi093 2023-12-14T18:20:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:20:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi093 to 17113M 2023-12-14T18:20:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:46.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:46.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:46.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:46.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:20:46.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:46 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:20:47.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2023-12-14T18:20:47.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:47.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi093 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi093 to 17113M 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:20:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:46 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:20:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 -- 172.21.15.97:0/2016355991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 msgr2=0x7f3750100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 --2- 172.21.15.97:0/2016355991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 0x7f3750100830 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3744009990 tx=0x7f3744009ca0 comp rx=0 tx=0).stop 2023-12-14T18:20:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 -- 172.21.15.97:0/2016355991 shutdown_connections 2023-12-14T18:20:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 --2- 172.21.15.97:0/2016355991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3750100d70 0x7f3750107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 --2- 172.21.15.97:0/2016355991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37500ff1e0 0x7f37500ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 --2- 172.21.15.97:0/2016355991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 0x7f3750100830 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 -- 172.21.15.97:0/2016355991 >> 172.21.15.97:0/2016355991 conn(0x7f37500fa7f0 msgr2=0x7f37500fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:20:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 -- 172.21.15.97:0/2016355991 shutdown_connections 2023-12-14T18:20:47.842 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 -- 172.21.15.97:0/2016355991 wait complete. 2023-12-14T18:20:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.841+0000 7f3754a94700 1 Processor -- start 2023-12-14T18:20:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f3754a94700 1 -- start start 2023-12-14T18:20:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f3754a94700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37500ff1e0 0x7f37501960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:47.844 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f3754a94700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 0x7f3750196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f3754a94700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3750100d70 0x7f375019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f3754a94700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375019dc50 con 0x7f3750100d70 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f3754a94700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375019dd90 con 0x7f37501003e0 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f3754a94700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f375019df20 con 0x7f37500ff1e0 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 0x7f3750196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 0x7f3750196600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:46550/0 (socket says 172.21.15.97:46550) 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374dd9b700 1 -- 172.21.15.97:0/1757383550 learned_addr learned my addr 172.21.15.97:0/1757383550 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374ed9d700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3750100d70 0x7f375019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374e59c700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37500ff1e0 0x7f37501960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374dd9b700 1 -- 172.21.15.97:0/1757383550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37500ff1e0 msgr2=0x7f37501960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374dd9b700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37500ff1e0 0x7f37501960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374dd9b700 1 -- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3750100d70 msgr2=0x7f375019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374dd9b700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3750100d70 0x7f375019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374dd9b700 1 -- 172.21.15.97:0/1757383550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37440095f0 con 0x7f37501003e0 2023-12-14T18:20:47.845 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374e59c700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37500ff1e0 0x7f37501960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:20:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f374dd9b700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 0x7f3750196600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f37440095c0 tx=0x7f374400f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.842+0000 7f373f7fe700 1 -- 172.21.15.97:0/1757383550 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374401d070 con 0x7f37501003e0 2023-12-14T18:20:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.843+0000 7f373f7fe700 1 -- 172.21.15.97:0/1757383550 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3744022470 con 0x7f37501003e0 2023-12-14T18:20:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.843+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f375010ab80 con 0x7f37501003e0 2023-12-14T18:20:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.843+0000 7f373f7fe700 1 -- 172.21.15.97:0/1757383550 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744017650 con 0x7f37501003e0 2023-12-14T18:20:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.843+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f375010a9f0 con 0x7f37501003e0 2023-12-14T18:20:47.846 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.844+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f375019e4b0 con 0x7f37501003e0 2023-12-14T18:20:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.845+0000 7f373f7fe700 1 -- 172.21.15.97:0/1757383550 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f3744022890 con 0x7f37501003e0 2023-12-14T18:20:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.845+0000 7f373f7fe700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f37380a10c0 0x7f37380a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:20:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.845+0000 7f373f7fe700 1 -- 172.21.15.97:0/1757383550 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1487+0+0 (secure 0 0 0) 0x7f37440cc1f0 con 0x7f37501003e0 2023-12-14T18:20:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.845+0000 7f374e59c700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f37380a10c0 0x7f37380a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:20:47.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.846+0000 7f374e59c700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f37380a10c0 0x7f37380a3570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3740005fd0 tx=0x7f3740005f00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:20:47.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:47.850+0000 7f373f7fe700 1 -- 172.21.15.97:0/1757383550 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3744097680 con 0x7f37501003e0 2023-12-14T18:20:47.930 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:47 smithi097 ceph-mon[43486]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-14T18:20:47.930 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:47 smithi097 ceph-mon[43486]: osdmap e6: 1 total, 0 up, 1 in 2023-12-14T18:20:47.930 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:47 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:47.930 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:47 smithi097 ceph-mon[43486]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:20:47.995 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:47 smithi160 ceph-mon[43840]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-14T18:20:47.995 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:47 smithi160 ceph-mon[43840]: osdmap e6: 1 total, 0 up, 1 in 2023-12-14T18:20:47.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:47 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:47.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:47 smithi160 ceph-mon[43840]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:20:48.045 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:20:48.045+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f375019eb20 con 0x7f37380a10c0 2023-12-14T18:20:48.065 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:20:47 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-0[54878]: 2023-12-14T18:20:47.703+0000 7f08b6e55700 -1 osd.0 0 waiting for initial osdmap 2023-12-14T18:20:48.065 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:20:47 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-0[54878]: 2023-12-14T18:20:47.710+0000 7f08af443700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:20:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:47 smithi093 ceph-mon[42070]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2023-12-14T18:20:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:47 smithi093 ceph-mon[42070]: osdmap e6: 1 total, 0 up, 1 in 2023-12-14T18:20:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:47 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:47 smithi093 ceph-mon[42070]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:20:48.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:48 smithi097 ceph-mon[43486]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:20:48.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:48 smithi097 ceph-mon[43486]: osdmap e7: 1 total, 0 up, 1 in 2023-12-14T18:20:48.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:48.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:48.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:48 smithi097 ceph-mon[43486]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:48.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:48 smithi097 ceph-mon[43486]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:48.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:20:48.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:20:48.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:48.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:48 smithi160 ceph-mon[43840]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:20:48.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:48 smithi160 ceph-mon[43840]: osdmap e7: 1 total, 0 up, 1 in 2023-12-14T18:20:48.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:48.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:48.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:48 smithi160 ceph-mon[43840]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:48.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:48 smithi160 ceph-mon[43840]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:48.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:20:48.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:20:48.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:49.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:48 smithi093 ceph-mon[42070]: from='osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:20:49.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:48 smithi093 ceph-mon[42070]: osdmap e7: 1 total, 0 up, 1 in 2023-12-14T18:20:49.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:49.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:49.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:48 smithi093 ceph-mon[42070]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2023-12-14T18:20:49.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:48 smithi093 ceph-mon[42070]: from='client.24137 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:20:49.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:20:49.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:20:49.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:49 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:20:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:49 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:20:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:49 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:49 smithi160 ceph-mon[43840]: osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] boot 2023-12-14T18:20:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:49 smithi160 ceph-mon[43840]: osdmap e8: 1 total, 1 up, 1 in 2023-12-14T18:20:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:49 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:50.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:49 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:20:50.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:49 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:20:50.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:49 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:50.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:49 smithi093 ceph-mon[42070]: osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] boot 2023-12-14T18:20:50.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:49 smithi093 ceph-mon[42070]: osdmap e8: 1 total, 1 up, 1 in 2023-12-14T18:20:50.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:49 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:49 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:20:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:49 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:20:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:49 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:49 smithi097 ceph-mon[43486]: osd.0 [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] boot 2023-12-14T18:20:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:49 smithi097 ceph-mon[43486]: osdmap e8: 1 total, 1 up, 1 in 2023-12-14T18:20:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:49 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:20:50.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:50 smithi160 ceph-mon[43840]: osdmap e9: 1 total, 1 up, 1 in 2023-12-14T18:20:50.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:50 smithi160 ceph-mon[43840]: pgmap v36: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:50.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:50 smithi160 ceph-mon[43840]: from='client.? 172.21.15.97:0/3076725322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2968fe6-d55f-4ae4-a2bc-e06b379812ae"}]: dispatch 2023-12-14T18:20:50.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:50 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2968fe6-d55f-4ae4-a2bc-e06b379812ae"}]: dispatch 2023-12-14T18:20:50.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:50 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2968fe6-d55f-4ae4-a2bc-e06b379812ae"}]': finished 2023-12-14T18:20:50.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:50 smithi160 ceph-mon[43840]: osdmap e10: 2 total, 1 up, 2 in 2023-12-14T18:20:50.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:20:50.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:50 smithi160 ceph-mon[43840]: from='client.? 172.21.15.97:0/2195247072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:20:51.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:50 smithi093 ceph-mon[42070]: osdmap e9: 1 total, 1 up, 1 in 2023-12-14T18:20:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:50 smithi093 ceph-mon[42070]: pgmap v36: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:50 smithi093 ceph-mon[42070]: from='client.? 172.21.15.97:0/3076725322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2968fe6-d55f-4ae4-a2bc-e06b379812ae"}]: dispatch 2023-12-14T18:20:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:50 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2968fe6-d55f-4ae4-a2bc-e06b379812ae"}]: dispatch 2023-12-14T18:20:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:50 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2968fe6-d55f-4ae4-a2bc-e06b379812ae"}]': finished 2023-12-14T18:20:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:50 smithi093 ceph-mon[42070]: osdmap e10: 2 total, 1 up, 2 in 2023-12-14T18:20:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:20:51.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:50 smithi093 ceph-mon[42070]: from='client.? 172.21.15.97:0/2195247072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:20:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:50 smithi097 ceph-mon[43486]: osdmap e9: 1 total, 1 up, 1 in 2023-12-14T18:20:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:50 smithi097 ceph-mon[43486]: pgmap v36: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:50 smithi097 ceph-mon[43486]: from='client.? 172.21.15.97:0/3076725322' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2968fe6-d55f-4ae4-a2bc-e06b379812ae"}]: dispatch 2023-12-14T18:20:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:50 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a2968fe6-d55f-4ae4-a2bc-e06b379812ae"}]: dispatch 2023-12-14T18:20:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:50 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a2968fe6-d55f-4ae4-a2bc-e06b379812ae"}]': finished 2023-12-14T18:20:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:50 smithi097 ceph-mon[43486]: osdmap e10: 2 total, 1 up, 2 in 2023-12-14T18:20:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:20:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:50 smithi097 ceph-mon[43486]: from='client.? 172.21.15.97:0/2195247072' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:20:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:52 smithi160 ceph-mon[43840]: pgmap v38: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:53.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:52 smithi093 ceph-mon[42070]: pgmap v38: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:53.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:52 smithi097 ceph-mon[43486]: pgmap v38: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:55.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:54 smithi097 ceph-mon[43486]: pgmap v39: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:55.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:54 smithi160 ceph-mon[43840]: pgmap v39: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:55.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:54 smithi093 ceph-mon[42070]: pgmap v39: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:56.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:56 smithi160 ceph-mon[43840]: pgmap v40: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:56.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:56 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-14T18:20:56.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:56 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:56.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:56 smithi160 ceph-mon[43840]: Deploying daemon osd.1 on smithi097 2023-12-14T18:20:57.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:56 smithi093 ceph-mon[42070]: pgmap v40: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:57.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:56 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-14T18:20:57.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:56 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:57.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:56 smithi093 ceph-mon[42070]: Deploying daemon osd.1 on smithi097 2023-12-14T18:20:57.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:56 smithi097 ceph-mon[43486]: pgmap v40: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:57.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:56 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2023-12-14T18:20:57.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:56 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:20:57.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:56 smithi097 ceph-mon[43486]: Deploying daemon osd.1 on smithi097 2023-12-14T18:20:59.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:20:58 smithi097 ceph-mon[43486]: pgmap v41: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:59.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:20:58 smithi160 ceph-mon[43840]: pgmap v41: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:20:59.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:20:58 smithi093 ceph-mon[42070]: pgmap v41: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:01.062 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:00 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:01.062 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:00 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:01.063 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:00 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:01.063 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:00 smithi097 ceph-mon[43486]: pgmap v42: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:01.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:00 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:01.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:00 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:00 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:00 smithi093 ceph-mon[42070]: pgmap v42: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:01.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:00 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:01.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:00 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:01.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:00 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:01.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:00 smithi160 ceph-mon[43840]: pgmap v42: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:02.049 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.048+0000 7f373f7fe700 1 -- 172.21.15.97:0/1757383550 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f375019eb20 con 0x7f37380a10c0 2023-12-14T18:21:02.050 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 1 on host 'smithi097' 2023-12-14T18:21:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f37380a10c0 msgr2=0x7f37380a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f37380a10c0 0x7f37380a3570 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3740005fd0 tx=0x7f3740005f00 comp rx=0 tx=0).stop 2023-12-14T18:21:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 msgr2=0x7f3750196600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 0x7f3750196600 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f37440095c0 tx=0x7f374400f690 comp rx=0 tx=0).stop 2023-12-14T18:21:02.053 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 shutdown_connections 2023-12-14T18:21:02.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f37380a10c0 0x7f37380a3570 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:02.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3750100d70 0x7f375019d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:02.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f37500ff1e0 0x7f37501960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:02.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 --2- 172.21.15.97:0/1757383550 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f37501003e0 0x7f3750196600 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:02.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 >> 172.21.15.97:0/1757383550 conn(0x7f37500fa7f0 msgr2=0x7f3750068630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:02.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 shutdown_connections 2023-12-14T18:21:02.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:02.052+0000 7f3754a94700 1 -- 172.21.15.97:0/1757383550 wait complete. 2023-12-14T18:21:02.439 DEBUG:teuthology.orchestra.run.smithi097:osd.1> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.1.service 2023-12-14T18:21:02.442 INFO:tasks.cephadm:Deploying osd.2 on smithi160 with /dev/vg_nvme/lv_4... 2023-12-14T18:21:02.442 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2023-12-14T18:21:02.475 INFO:journalctl@ceph.osd.1.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:21:02.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:02.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:02.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:02 smithi160 ceph-mon[43840]: pgmap v43: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:02.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:02.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:02.761 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:21:02.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:02.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:02.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:02 smithi097 ceph-mon[43486]: pgmap v43: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:02.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:02.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:03.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:03.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:03.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:02 smithi093 ceph-mon[42070]: pgmap v43: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:03.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:03.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:04.027 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_4 2023-12-14T18:21:04.027 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2023-12-14T18:21:04.027 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2023-12-14T18:21:04.027 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2023-12-14T18:21:04.027 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111885 s, 937 MB/s 2023-12-14T18:21:04.028 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2023-12-14T18:21:04.036 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:03 smithi097 ceph-mon[43486]: from='osd.1 [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:21:04.036 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:03 smithi097 ceph-mon[43486]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:21:04.036 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-1[48411]: 2023-12-14T18:21:03.775+0000 7fb24880d640 -1 osd.1 0 log_to_monitors true 2023-12-14T18:21:04.194 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:03 smithi160 ceph-mon[43840]: from='osd.1 [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:21:04.195 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:03 smithi160 ceph-mon[43840]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:21:04.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:03 smithi093 ceph-mon[42070]: from='osd.1 [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:21:04.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:03 smithi093 ceph-mon[42070]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2023-12-14T18:21:04.415 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi160:vg_nvme/lv_4 2023-12-14T18:21:04.725 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:21:05.155 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-14T18:21:05.155 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: osdmap e11: 2 total, 1 up, 2 in 2023-12-14T18:21:05.155 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='osd.1 [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: pgmap v45: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi097 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi097 to 17114M 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:05.156 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:05.157 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:05.157 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.157 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:21:05.157 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:05.157 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:05.157 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:05.157 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:05.157 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-14T18:21:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: osdmap e11: 2 total, 1 up, 2 in 2023-12-14T18:21:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='osd.1 [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: pgmap v45: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi097 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi097 to 17114M 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:21:05.317 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:05.317 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:05.317 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:05.317 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:05.317 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.342 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-1[48411]: 2023-12-14T18:21:04.846+0000 7fb23de84700 -1 osd.1 0 waiting for initial osdmap 2023-12-14T18:21:05.343 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-1[48411]: 2023-12-14T18:21:04.849+0000 7fb239478700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:21:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2023-12-14T18:21:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: osdmap e11: 2 total, 1 up, 2 in 2023-12-14T18:21:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='osd.1 [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: pgmap v45: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi097 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi097 to 17114M 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:21:05.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:05.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:05.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:05.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:05.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:05.676 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.672+0000 7f1f8af2e700 1 -- 172.21.15.160:0/2834162198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 msgr2=0x7f1f840ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:05.677 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.672+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/2834162198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 0x7f1f840ff5f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f1f780098f0 tx=0x7f1f78009c00 comp rx=0 tx=0).stop 2023-12-14T18:21:05.677 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.672+0000 7f1f8af2e700 1 -- 172.21.15.160:0/2834162198 shutdown_connections 2023-12-14T18:21:05.677 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.672+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/2834162198 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f84100d70 0x7f1f84107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:05.677 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.672+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/2834162198 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 0x7f1f840ff5f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:05.677 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.672+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/2834162198 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f841003e0 0x7f1f84100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:05.677 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.672+0000 7f1f8af2e700 1 -- 172.21.15.160:0/2834162198 >> 172.21.15.160:0/2834162198 conn(0x7f1f840fa7f0 msgr2=0x7f1f840fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:05.677 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.672+0000 7f1f8af2e700 1 -- 172.21.15.160:0/2834162198 shutdown_connections 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.672+0000 7f1f8af2e700 1 -- 172.21.15.160:0/2834162198 wait complete. 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f8af2e700 1 Processor -- start 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f8af2e700 1 -- start start 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f8af2e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 0x7f1f84069110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f8af2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f841003e0 0x7f1f8419d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f8af2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f84100d70 0x7f1f8419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f8af2e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419fe40 con 0x7f1f841003e0 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f8af2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f8419ffb0 con 0x7f1f84100d70 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f8af2e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f841a0170 con 0x7f1f840ff1e0 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f88cca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 0x7f1f84069110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:05.678 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f88cca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 0x7f1f84069110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:55236/0 (socket says 172.21.15.160:55236) 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.673+0000 7f1f88cca700 1 -- 172.21.15.160:0/3137720609 learned_addr learned my addr 172.21.15.160:0/3137720609 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f83fff700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f841003e0 0x7f1f8419d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f894cb700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f84100d70 0x7f1f8419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f88cca700 1 -- 172.21.15.160:0/3137720609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f84100d70 msgr2=0x7f1f8419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f88cca700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f84100d70 0x7f1f8419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f88cca700 1 -- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f841003e0 msgr2=0x7f1f8419d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f88cca700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f841003e0 0x7f1f8419d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f88cca700 1 -- 172.21.15.160:0/3137720609 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f78009520 con 0x7f1f840ff1e0 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f894cb700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f84100d70 0x7f1f8419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f88cca700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 0x7f1f84069110 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009fd0 tx=0x7f1f7800b610 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:05.679 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f81ffb700 1 -- 172.21.15.160:0/3137720609 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f7801c070 con 0x7f1f840ff1e0 2023-12-14T18:21:05.680 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f81ffb700 1 -- 172.21.15.160:0/3137720609 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1f78005040 con 0x7f1f840ff1e0 2023-12-14T18:21:05.680 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f8410ab90 con 0x7f1f840ff1e0 2023-12-14T18:21:05.680 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f81ffb700 1 -- 172.21.15.160:0/3137720609 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f78004bd0 con 0x7f1f840ff1e0 2023-12-14T18:21:05.680 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.674+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f8410aa00 con 0x7f1f840ff1e0 2023-12-14T18:21:05.680 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.675+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f841a0760 con 0x7f1f840ff1e0 2023-12-14T18:21:05.681 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.676+0000 7f1f81ffb700 1 -- 172.21.15.160:0/3137720609 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7f1f78020070 con 0x7f1f840ff1e0 2023-12-14T18:21:05.681 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.677+0000 7f1f81ffb700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1f740a11c0 0x7f1f740a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:05.681 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.677+0000 7f1f81ffb700 1 -- 172.21.15.160:0/3137720609 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2029+0+0 (secure 0 0 0) 0x7f1f780cbdf0 con 0x7f1f840ff1e0 2023-12-14T18:21:05.681 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.677+0000 7f1f83fff700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1f740a11c0 0x7f1f740a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:05.681 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.677+0000 7f1f83fff700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1f740a11c0 0x7f1f740a3670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f84069690 tx=0x7f1f70009310 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:05.682 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.681+0000 7f1f81ffb700 1 -- 172.21.15.160:0/3137720609 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1f78096ea0 con 0x7f1f840ff1e0 2023-12-14T18:21:05.878 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:05.876+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7f1f84060f30 con 0x7f1f740a11c0 2023-12-14T18:21:06.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:05 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:06.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:05 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:21:06.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:05 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:06.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:05 smithi160 ceph-mon[43840]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:21:06.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:05 smithi160 ceph-mon[43840]: osdmap e12: 2 total, 1 up, 2 in 2023-12-14T18:21:06.248 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:05 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:06.248 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:05 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:06.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:05 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:06.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:05 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:21:06.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:05 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:06.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:05 smithi093 ceph-mon[42070]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:21:06.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:05 smithi093 ceph-mon[42070]: osdmap e12: 2 total, 1 up, 2 in 2023-12-14T18:21:06.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:05 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:06.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:05 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:06.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:05 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:06.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:05 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:21:06.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:05 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:06.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:05 smithi097 ceph-mon[43486]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:21:06.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:05 smithi097 ceph-mon[43486]: osdmap e12: 2 total, 1 up, 2 in 2023-12-14T18:21:06.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:05 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:06.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:05 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:06.926 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:21:06.926 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:21:06.927 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:06.927 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: osd.1 [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] boot 2023-12-14T18:21:06.927 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: osdmap e13: 2 total, 2 up, 2 in 2023-12-14T18:21:06.927 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:06.927 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:06.927 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:06.927 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:06.927 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:06.927 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:06 smithi160 ceph-mon[43840]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:21:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:21:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: osd.1 [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] boot 2023-12-14T18:21:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: osdmap e13: 2 total, 2 up, 2 in 2023-12-14T18:21:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:07.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:07.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:07.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:06 smithi093 ceph-mon[42070]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:21:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:21:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: osd.1 [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] boot 2023-12-14T18:21:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: osdmap e13: 2 total, 2 up, 2 in 2023-12-14T18:21:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:21:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: from='client.24148 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:07.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:07.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:06 smithi097 ceph-mon[43486]: pgmap v48: 0 pgs: ; 0 B data, 4.9 MiB used, 89 GiB / 89 GiB avail 2023-12-14T18:21:08.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:07 smithi160 ceph-mon[43840]: osdmap e14: 2 total, 2 up, 2 in 2023-12-14T18:21:08.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:07 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/2921033255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9138f66-3120-4842-af4c-40a6c4c57710"}]: dispatch 2023-12-14T18:21:08.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:07 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9138f66-3120-4842-af4c-40a6c4c57710"}]: dispatch 2023-12-14T18:21:08.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:07 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9138f66-3120-4842-af4c-40a6c4c57710"}]': finished 2023-12-14T18:21:08.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:07 smithi160 ceph-mon[43840]: osdmap e15: 3 total, 2 up, 3 in 2023-12-14T18:21:08.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:07 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:08.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:07 smithi093 ceph-mon[42070]: osdmap e14: 2 total, 2 up, 2 in 2023-12-14T18:21:08.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:07 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/2921033255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9138f66-3120-4842-af4c-40a6c4c57710"}]: dispatch 2023-12-14T18:21:08.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:07 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9138f66-3120-4842-af4c-40a6c4c57710"}]: dispatch 2023-12-14T18:21:08.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:07 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9138f66-3120-4842-af4c-40a6c4c57710"}]': finished 2023-12-14T18:21:08.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:07 smithi093 ceph-mon[42070]: osdmap e15: 3 total, 2 up, 3 in 2023-12-14T18:21:08.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:07 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:08.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:07 smithi097 ceph-mon[43486]: osdmap e14: 2 total, 2 up, 2 in 2023-12-14T18:21:08.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:07 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/2921033255' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9138f66-3120-4842-af4c-40a6c4c57710"}]: dispatch 2023-12-14T18:21:08.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:07 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a9138f66-3120-4842-af4c-40a6c4c57710"}]: dispatch 2023-12-14T18:21:08.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:07 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a9138f66-3120-4842-af4c-40a6c4c57710"}]': finished 2023-12-14T18:21:08.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:07 smithi097 ceph-mon[43486]: osdmap e15: 3 total, 2 up, 3 in 2023-12-14T18:21:08.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:07 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:09.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:08 smithi160 ceph-mon[43840]: pgmap v51: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:09.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:08 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/842332982' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:09.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:08 smithi093 ceph-mon[42070]: pgmap v51: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:09.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:08 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/842332982' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:09.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:08 smithi097 ceph-mon[43486]: pgmap v51: 0 pgs: ; 0 B data, 9.8 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:09.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:08 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/842332982' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:11.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:10 smithi160 ceph-mon[43840]: pgmap v52: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:11.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:10 smithi093 ceph-mon[42070]: pgmap v52: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:11.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:10 smithi097 ceph-mon[43486]: pgmap v52: 0 pgs: ; 0 B data, 9.9 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:13.172 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:12 smithi160 ceph-mon[43840]: pgmap v53: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:13.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:12 smithi093 ceph-mon[42070]: pgmap v53: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:13.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:12 smithi097 ceph-mon[43486]: pgmap v53: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:14.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-14T18:21:14.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:14.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-14T18:21:14.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:14.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2023-12-14T18:21:14.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:15.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:14 smithi160 ceph-mon[43840]: Deploying daemon osd.2 on smithi160 2023-12-14T18:21:15.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:14 smithi160 ceph-mon[43840]: pgmap v54: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:15.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:14 smithi093 ceph-mon[42070]: Deploying daemon osd.2 on smithi160 2023-12-14T18:21:15.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:14 smithi093 ceph-mon[42070]: pgmap v54: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:15.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:14 smithi097 ceph-mon[43486]: Deploying daemon osd.2 on smithi160 2023-12-14T18:21:15.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:14 smithi097 ceph-mon[43486]: pgmap v54: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:16.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:16 smithi160 ceph-mon[43840]: pgmap v55: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:17.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:16 smithi093 ceph-mon[42070]: pgmap v55: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:17.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:16 smithi097 ceph-mon[43486]: pgmap v55: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:18.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:19.445 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:19 smithi160 ceph-mon[43840]: pgmap v56: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:19.446 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:19 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:19.446 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:19.446 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:19 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:21:19.446 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:19 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:19 smithi093 ceph-mon[42070]: pgmap v56: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:19 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:19 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:21:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:19 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:19.580 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.578+0000 7f1f81ffb700 1 -- 172.21.15.160:0/3137720609 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1f84060f30 con 0x7f1f740a11c0 2023-12-14T18:21:19.580 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 2 on host 'smithi160' 2023-12-14T18:21:19.582 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1f740a11c0 msgr2=0x7f1f740a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:19.582 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1f740a11c0 0x7f1f740a3670 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1f84069690 tx=0x7f1f70009310 comp rx=0 tx=0).stop 2023-12-14T18:21:19.582 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 msgr2=0x7f1f84069110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:19.582 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 0x7f1f84069110 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009fd0 tx=0x7f1f7800b610 comp rx=0 tx=0).stop 2023-12-14T18:21:19.582 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 shutdown_connections 2023-12-14T18:21:19.582 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1f740a11c0 0x7f1f740a3670 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:19.583 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1f84100d70 0x7f1f8419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:19.583 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1f840ff1e0 0x7f1f84069110 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:19.583 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 --2- 172.21.15.160:0/3137720609 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1f841003e0 0x7f1f8419d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:19.583 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.580+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 >> 172.21.15.160:0/3137720609 conn(0x7f1f840fa7f0 msgr2=0x7f1f840fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:19.583 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.581+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 shutdown_connections 2023-12-14T18:21:19.583 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:19.581+0000 7f1f8af2e700 1 -- 172.21.15.160:0/3137720609 wait complete. 2023-12-14T18:21:19.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:19 smithi097 ceph-mon[43486]: pgmap v56: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:19 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:19 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:21:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:19 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:19.951 DEBUG:teuthology.orchestra.run.smithi160:osd.2> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.2.service 2023-12-14T18:21:19.953 INFO:tasks.cephadm:Deploying osd.3 on smithi093 with /dev/vg_nvme/lv_3... 2023-12-14T18:21:19.954 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-14T18:21:19.989 INFO:journalctl@ceph.osd.2.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:21:20.275 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:20.713 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:20.713 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:20.713 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:20.713 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:20 smithi093 ceph-mon[42070]: pgmap v57: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:20.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:20.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:20.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:20.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:20 smithi160 ceph-mon[43840]: pgmap v57: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:20.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:20.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:20.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:20.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:20 smithi097 ceph-mon[43486]: pgmap v57: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:21.379 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-2[49563]: 2023-12-14T18:21:21.107+0000 7f1eec570640 -1 osd.2 0 log_to_monitors true 2023-12-14T18:21:21.480 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-14T18:21:21.480 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-14T18:21:21.480 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-12-14T18:21:21.481 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-12-14T18:21:21.481 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110628 s, 948 MB/s 2023-12-14T18:21:21.481 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-12-14T18:21:21.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:21 smithi093 ceph-mon[42070]: from='osd.2 [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:21:21.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:21 smithi093 ceph-mon[42070]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:21:21.901 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi093:vg_nvme/lv_3 2023-12-14T18:21:21.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:21 smithi160 ceph-mon[43840]: from='osd.2 [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:21:21.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:21 smithi160 ceph-mon[43840]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:21:22.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:21 smithi097 ceph-mon[43486]: from='osd.2 [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:21:22.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:21 smithi097 ceph-mon[43486]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2023-12-14T18:21:22.209 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:21:22.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-14T18:21:22.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: osdmap e16: 3 total, 2 up, 3 in 2023-12-14T18:21:22.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:22.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='osd.2 [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:21:22.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:21:22.940 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi160 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi160 to 17114M 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:22.941 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:22 smithi093 ceph-mon[42070]: pgmap v59: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:22.996 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-2[49563]: 2023-12-14T18:21:22.711+0000 7f1ee1be7700 -1 osd.2 0 waiting for initial osdmap 2023-12-14T18:21:22.996 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-2[49563]: 2023-12-14T18:21:22.718+0000 7f1edb9d8700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:21:22.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-14T18:21:22.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: osdmap e16: 3 total, 2 up, 3 in 2023-12-14T18:21:22.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:22.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='osd.2 [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:21:22.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi160 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi160 to 17114M 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:22.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:22 smithi160 ceph-mon[43840]: pgmap v59: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:23.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2023-12-14T18:21:23.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: osdmap e16: 3 total, 2 up, 3 in 2023-12-14T18:21:23.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:23.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='osd.2 [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:21:23.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:21:23.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi160 2023-12-14T18:21:23.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:23.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:23.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:23.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi160 to 17114M 2023-12-14T18:21:23.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:23.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:23.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:23.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:23.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:23.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:21:23.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:23.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:22 smithi097 ceph-mon[43486]: pgmap v59: 0 pgs: ; 0 B data, 10 MiB used, 179 GiB / 179 GiB avail 2023-12-14T18:21:23.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.117+0000 7fb2f3368700 1 -- 172.21.15.93:0/1905082936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec0ff1e0 msgr2=0x7fb2ec0ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:23.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.117+0000 7fb2f3368700 1 --2- 172.21.15.93:0/1905082936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec0ff1e0 0x7fb2ec0ff5f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80098f0 tx=0x7fb2e8009c00 comp rx=0 tx=0).stop 2023-12-14T18:21:23.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.117+0000 7fb2f3368700 1 -- 172.21.15.93:0/1905082936 shutdown_connections 2023-12-14T18:21:23.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.117+0000 7fb2f3368700 1 --2- 172.21.15.93:0/1905082936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec0ff1e0 0x7fb2ec0ff5f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:23.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.117+0000 7fb2f3368700 1 --2- 172.21.15.93:0/1905082936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec100d70 0x7fb2ec107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:23.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.117+0000 7fb2f3368700 1 --2- 172.21.15.93:0/1905082936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2ec1003e0 0x7fb2ec100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:23.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.117+0000 7fb2f3368700 1 -- 172.21.15.93:0/1905082936 >> 172.21.15.93:0/1905082936 conn(0x7fb2ec0fa7f0 msgr2=0x7fb2ec0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:23.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.117+0000 7fb2f3368700 1 -- 172.21.15.93:0/1905082936 shutdown_connections 2023-12-14T18:21:23.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.117+0000 7fb2f3368700 1 -- 172.21.15.93:0/1905082936 wait complete. 2023-12-14T18:21:23.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f3368700 1 Processor -- start 2023-12-14T18:21:23.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f3368700 1 -- start start 2023-12-14T18:21:23.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f3368700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2ec0ff1e0 0x7fb2ec0699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:23.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f3368700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec1003e0 0x7fb2ec19adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:23.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f3368700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec100d70 0x7fb2ec19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:23.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f3368700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec19db90 con 0x7fb2ec100d70 2023-12-14T18:21:23.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f3368700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec19dd00 con 0x7fb2ec1003e0 2023-12-14T18:21:23.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f3368700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2ec19dec0 con 0x7fb2ec0ff1e0 2023-12-14T18:21:23.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f1905700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec100d70 0x7fb2ec19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:23.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f1905700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec100d70 0x7fb2ec19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33000/0 (socket says 172.21.15.93:33000) 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.118+0000 7fb2f1905700 1 -- 172.21.15.93:0/757806873 learned_addr learned my addr 172.21.15.93:0/757806873 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f0903700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec1003e0 0x7fb2ec19adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f1104700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2ec0ff1e0 0x7fb2ec0699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f1905700 1 -- 172.21.15.93:0/757806873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2ec0ff1e0 msgr2=0x7fb2ec0699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f1905700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2ec0ff1e0 0x7fb2ec0699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f1905700 1 -- 172.21.15.93:0/757806873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec1003e0 msgr2=0x7fb2ec19adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f1905700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec1003e0 0x7fb2ec19adb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f1905700 1 -- 172.21.15.93:0/757806873 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2dc0095f0 con 0x7fb2ec100d70 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f1905700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec100d70 0x7fb2ec19d3f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c370 tx=0x7fb2e400c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:23.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2e27fc700 1 -- 172.21.15.93:0/757806873 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e4017070 con 0x7fb2ec100d70 2023-12-14T18:21:23.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2e4009c40 con 0x7fb2ec100d70 2023-12-14T18:21:23.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2e27fc700 1 -- 172.21.15.93:0/757806873 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb2e400f040 con 0x7fb2ec100d70 2023-12-14T18:21:23.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2e8009520 con 0x7fb2ec100d70 2023-12-14T18:21:23.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.119+0000 7fb2e27fc700 1 -- 172.21.15.93:0/757806873 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2e400e050 con 0x7fb2ec100d70 2023-12-14T18:21:23.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.120+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2ec10a9a0 con 0x7fb2ec100d70 2023-12-14T18:21:23.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.122+0000 7fb2e27fc700 1 -- 172.21.15.93:0/757806873 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 15) v1 ==== 135659+0+0 (secure 0 0 0) 0x7fb2e40044c0 con 0x7fb2ec100d70 2023-12-14T18:21:23.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.122+0000 7fb2e27fc700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb2d80a10c0 0x7fb2d80a3570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:23.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.122+0000 7fb2f1104700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb2d80a10c0 0x7fb2d80a3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:23.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.123+0000 7fb2e27fc700 1 -- 172.21.15.93:0/757806873 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 2571+0+0 (secure 0 0 0) 0x7fb2e40cb6d0 con 0x7fb2ec100d70 2023-12-14T18:21:23.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.123+0000 7fb2f1104700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb2d80a10c0 0x7fb2d80a3570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80098c0 tx=0x7fb2e8019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:23.126 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.126+0000 7fb2e27fc700 1 -- 172.21.15.93:0/757806873 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb2e40964d0 con 0x7fb2ec100d70 2023-12-14T18:21:23.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:23.324+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2ec060f30 con 0x7fb2d80a10c0 2023-12-14T18:21:23.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:23 smithi160 ceph-mon[43840]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:21:23.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:23 smithi160 ceph-mon[43840]: osdmap e17: 3 total, 2 up, 3 in 2023-12-14T18:21:23.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:23.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:23.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:23 smithi160 ceph-mon[43840]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:23.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:23.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:23.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:24.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:23 smithi093 ceph-mon[42070]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:21:24.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:23 smithi093 ceph-mon[42070]: osdmap e17: 3 total, 2 up, 3 in 2023-12-14T18:21:24.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:24.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:24.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:23 smithi093 ceph-mon[42070]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:24.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:24.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:24.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:24.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:23 smithi097 ceph-mon[43486]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:21:24.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:23 smithi097 ceph-mon[43486]: osdmap e17: 3 total, 2 up, 3 in 2023-12-14T18:21:24.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:24.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:24.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:23 smithi097 ceph-mon[43486]: from='client.14292 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:24.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:24.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:24.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:24.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:24 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:21:24.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:24 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:21:24.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:24 smithi160 ceph-mon[43840]: osd.2 [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] boot 2023-12-14T18:21:24.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:24 smithi160 ceph-mon[43840]: osdmap e18: 3 total, 3 up, 3 in 2023-12-14T18:21:24.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:24 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:24.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:24 smithi160 ceph-mon[43840]: pgmap v62: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:24.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:24 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:21:25.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:24 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:21:25.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:24 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:21:25.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:24 smithi093 ceph-mon[42070]: osd.2 [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] boot 2023-12-14T18:21:25.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:24 smithi093 ceph-mon[42070]: osdmap e18: 3 total, 3 up, 3 in 2023-12-14T18:21:25.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:24 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:25.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:24 smithi093 ceph-mon[42070]: pgmap v62: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:25.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:24 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:21:25.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:24 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:21:25.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:24 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:21:25.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:24 smithi097 ceph-mon[43486]: osd.2 [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] boot 2023-12-14T18:21:25.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:24 smithi097 ceph-mon[43486]: osdmap e18: 3 total, 3 up, 3 in 2023-12-14T18:21:25.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:24 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:21:25.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:24 smithi097 ceph-mon[43486]: pgmap v62: 0 pgs: ; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:25.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:24 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:21:25.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-14T18:21:25.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:25 smithi093 ceph-mon[42070]: osdmap e19: 3 total, 3 up, 3 in 2023-12-14T18:21:25.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:21:25.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:25 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2022038845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "317f1a16-4f77-46e5-aa43-63e8d930682b"}]: dispatch 2023-12-14T18:21:25.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:21:25.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:25 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2022038845' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "317f1a16-4f77-46e5-aa43-63e8d930682b"}]': finished 2023-12-14T18:21:25.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:25 smithi093 ceph-mon[42070]: osdmap e20: 4 total, 3 up, 4 in 2023-12-14T18:21:25.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:25.816 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:21:25 smithi093 sudo[58131]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:21:25.816 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:21:25 smithi093 sudo[58131]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:25.816 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:21:25 smithi093 sudo[58131]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:25.816 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:21:25 smithi093 sudo[58131]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:25.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-14T18:21:25.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:25 smithi160 ceph-mon[43840]: osdmap e19: 3 total, 3 up, 3 in 2023-12-14T18:21:25.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:21:25.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:25 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2022038845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "317f1a16-4f77-46e5-aa43-63e8d930682b"}]: dispatch 2023-12-14T18:21:25.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:21:25.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:25 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2022038845' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "317f1a16-4f77-46e5-aa43-63e8d930682b"}]': finished 2023-12-14T18:21:25.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:25 smithi160 ceph-mon[43840]: osdmap e20: 4 total, 3 up, 4 in 2023-12-14T18:21:25.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:26.077 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2023-12-14T18:21:26.077 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:25 smithi097 ceph-mon[43486]: osdmap e19: 3 total, 3 up, 3 in 2023-12-14T18:21:26.077 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:21:26.077 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:25 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2022038845' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "317f1a16-4f77-46e5-aa43-63e8d930682b"}]: dispatch 2023-12-14T18:21:26.077 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:21:26.078 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:25 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2022038845' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "317f1a16-4f77-46e5-aa43-63e8d930682b"}]': finished 2023-12-14T18:21:26.078 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:25 smithi097 ceph-mon[43486]: osdmap e20: 4 total, 3 up, 4 in 2023-12-14T18:21:26.078 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:26.314 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:21:25 smithi093 sudo[58154]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:21:26.315 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:21:25 smithi093 sudo[58154]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:26.315 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:21:25 smithi093 sudo[58154]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:26.315 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:21:25 smithi093 sudo[58154]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:26.342 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:26 smithi097 sudo[50799]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:21:26.343 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:26 smithi097 sudo[50799]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:26.343 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:26 smithi097 sudo[50799]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:26.343 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:26 smithi097 sudo[50799]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:26.343 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:26 smithi097 sudo[50802]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:21:26.343 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:26 smithi097 sudo[50802]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:26.343 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:26 smithi097 sudo[50802]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:26.343 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:21:26 smithi097 sudo[50802]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:26.837 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:26 smithi160 sudo[51683]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:21:26.838 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:26 smithi160 sudo[51683]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:26.838 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:26 smithi160 sudo[51683]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:26.838 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:26 smithi160 sudo[51683]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:27.099 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:27.100 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:26 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/60065549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:27.100 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:26 smithi093 ceph-mon[42070]: pgmap v65: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:27.100 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:26 smithi093 ceph-mon[42070]: osdmap e21: 4 total, 3 up, 4 in 2023-12-14T18:21:27.100 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:27.100 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:27.246 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:26 smithi160 sudo[51686]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:21:27.246 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:26 smithi160 sudo[51686]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:27.246 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:26 smithi160 sudo[51686]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:27.246 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:21:26 smithi160 sudo[51686]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:26 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/60065549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:26 smithi160 ceph-mon[43840]: pgmap v65: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:27.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:26 smithi160 ceph-mon[43840]: osdmap e21: 4 total, 3 up, 4 in 2023-12-14T18:21:27.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:27.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:27.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:27.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:26 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/60065549' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:27.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:26 smithi097 ceph-mon[43486]: pgmap v65: 1 pgs: 1 unknown; 0 B data, 15 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:27.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:26 smithi097 ceph-mon[43486]: osdmap e21: 4 total, 3 up, 4 in 2023-12-14T18:21:27.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:27.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:27.351 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:27 smithi093 sudo[58577]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:21:27.351 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:27 smithi093 sudo[58577]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:27.351 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:27 smithi093 sudo[58577]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:27.351 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:27 smithi093 sudo[58577]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:27.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:27.363+0000 7fb2e27fc700 1 -- 172.21.15.93:0/757806873 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb2e4004770 con 0x7fb2ec100d70 2023-12-14T18:21:27.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:27 smithi093 sudo[58765]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:21:27.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:27 smithi093 sudo[58765]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:27.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:27 smithi093 sudo[58765]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:27.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:27 smithi093 sudo[58765]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:27 smithi160 sudo[51689]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:21:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:27 smithi160 sudo[51689]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:27 smithi160 sudo[51689]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:27 smithi160 sudo[51689]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:27 smithi160 sudo[51692]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:21:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:27 smithi160 sudo[51692]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:27 smithi160 sudo[51692]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:27.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:27 smithi160 sudo[51692]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:28.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 sudo[50805]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/smartctl -x --json=o /dev/nvme0n1 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 sudo[50805]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 sudo[50805]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 sudo[50805]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 sudo[50808]: ceph : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 sudo[50808]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 sudo[50808]: pam_unix(sudo:session): session opened for user root by (uid=0) 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 sudo[50808]: pam_unix(sudo:session): session closed for user root 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:21:28.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: mgrmap e16: x(active, since 111s), standbys: z, y 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:28.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:28.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:21:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:21:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:28.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:21:28.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: mgrmap e16: x(active, since 111s), standbys: z, y 2023-12-14T18:21:28.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:28.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:28.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:28.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:21:28.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:28.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:28.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: mgrmap e16: x(active, since 111s), standbys: z, y 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:21:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:21:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:21:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:21:29.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:29 smithi093 ceph-mon[42070]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:21:29.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:29 smithi093 ceph-mon[42070]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 16 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:29.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:29 smithi093 ceph-mon[42070]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:21:29.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:29 smithi097 ceph-mon[43486]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:21:29.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:29 smithi097 ceph-mon[43486]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 16 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:29.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:29 smithi097 ceph-mon[43486]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:21:29.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:29 smithi160 ceph-mon[43840]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2023-12-14T18:21:29.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:29 smithi160 ceph-mon[43840]: pgmap v67: 1 pgs: 1 active+clean; 577 KiB data, 16 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:29.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:29 smithi160 ceph-mon[43840]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2023-12-14T18:21:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:31 smithi093 ceph-mon[42070]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-14T18:21:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:31 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:31.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:31 smithi097 ceph-mon[43486]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:31.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-14T18:21:31.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:31 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:31.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:31 smithi160 ceph-mon[43840]: pgmap v68: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail 2023-12-14T18:21:31.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2023-12-14T18:21:31.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:31 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:32.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:32 smithi097 ceph-mon[43486]: Deploying daemon osd.3 on smithi093 2023-12-14T18:21:32.706 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:32 smithi093 ceph-mon[42070]: Deploying daemon osd.3 on smithi093 2023-12-14T18:21:32.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:32 smithi160 ceph-mon[43840]: Deploying daemon osd.3 on smithi093 2023-12-14T18:21:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:33 smithi093 ceph-mon[42070]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-12-14T18:21:33.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:33 smithi097 ceph-mon[43486]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-12-14T18:21:33.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:33 smithi160 ceph-mon[43840]: pgmap v69: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 3.0 KiB/s rd, 179 KiB/s wr, 10 op/s 2023-12-14T18:21:35.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:35 smithi093 ceph-mon[42070]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 119 KiB/s wr, 7 op/s 2023-12-14T18:21:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:35.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:35 smithi097 ceph-mon[43486]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 119 KiB/s wr, 7 op/s 2023-12-14T18:21:35.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:35.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:35.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:35.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:35 smithi160 ceph-mon[43840]: pgmap v70: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 2.0 KiB/s rd, 119 KiB/s wr, 7 op/s 2023-12-14T18:21:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:37.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:37.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:37.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:37.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:37.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:36 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:37.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:36 smithi093 ceph-mon[42070]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 90 KiB/s wr, 5 op/s 2023-12-14T18:21:37.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:37.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:36 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:21:37.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:36 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:36 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:36 smithi160 ceph-mon[43840]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 90 KiB/s wr, 5 op/s 2023-12-14T18:21:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:36 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:21:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:36 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:37.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:37.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:37.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:36 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:36 smithi097 ceph-mon[43486]: pgmap v71: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.5 KiB/s rd, 90 KiB/s wr, 5 op/s 2023-12-14T18:21:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:36 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:21:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:36 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:37.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.368+0000 7fb2e27fc700 1 -- 172.21.15.93:0/757806873 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb2ec060f30 con 0x7fb2d80a10c0 2023-12-14T18:21:37.369 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 3 on host 'smithi093' 2023-12-14T18:21:37.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb2d80a10c0 msgr2=0x7fb2d80a3570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:37.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb2d80a10c0 0x7fb2d80a3570 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fb2e80098c0 tx=0x7fb2e8019040 comp rx=0 tx=0).stop 2023-12-14T18:21:37.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec100d70 msgr2=0x7fb2ec19d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec100d70 0x7fb2ec19d3f0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb2e400c370 tx=0x7fb2e400c680 comp rx=0 tx=0).stop 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 shutdown_connections 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb2d80a10c0 0x7fb2d80a3570 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb2ec100d70 0x7fb2ec19d3f0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb2ec0ff1e0 0x7fb2ec0699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 --2- 172.21.15.93:0/757806873 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb2ec1003e0 0x7fb2ec19adb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 >> 172.21.15.93:0/757806873 conn(0x7fb2ec0fa7f0 msgr2=0x7fb2ec0fca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 shutdown_connections 2023-12-14T18:21:37.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:21:37.370+0000 7fb2f3368700 1 -- 172.21.15.93:0/757806873 wait complete. 2023-12-14T18:21:37.759 DEBUG:teuthology.orchestra.run.smithi093:osd.3> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.3.service 2023-12-14T18:21:37.761 INFO:tasks.cephadm:Deploying osd.4 on smithi097 with /dev/vg_nvme/lv_3... 2023-12-14T18:21:37.761 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-14T18:21:37.793 INFO:journalctl@ceph.osd.3.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:21:38.078 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:21:38.464 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:38 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:38.464 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:38 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:38.464 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:38 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:38.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:38 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:38.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:38 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:38.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:38 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:38.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:38 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:38.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:38 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:38.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:38 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:39.065 INFO:journalctl@ceph.osd.3.smithi093.stdout:Dec 14 18:21:38 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-3[60361]: 2023-12-14T18:21:38.813+0000 7f0985a52640 -1 osd.3 0 log_to_monitors true 2023-12-14T18:21:39.398 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-14T18:21:39.398 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-14T18:21:39.399 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-12-14T18:21:39.399 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-12-14T18:21:39.399 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115955 s, 904 MB/s 2023-12-14T18:21:39.399 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-12-14T18:21:39.522 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:39 smithi093 ceph-mon[42070]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 4 op/s 2023-12-14T18:21:39.523 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:39 smithi093 ceph-mon[42070]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-14T18:21:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:39 smithi097 ceph-mon[43486]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 4 op/s 2023-12-14T18:21:39.615 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:39 smithi097 ceph-mon[43486]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-14T18:21:39.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:39 smithi160 ceph-mon[43840]: pgmap v72: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 4 op/s 2023-12-14T18:21:39.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:39 smithi160 ceph-mon[43840]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2023-12-14T18:21:39.753 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_3 2023-12-14T18:21:40.055 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:21:40.572 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-14T18:21:40.572 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: osdmap e22: 4 total, 3 up, 4 in 2023-12-14T18:21:40.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:40.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:21:40.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi093 2023-12-14T18:21:40.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:40.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:40.573 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi093 to 8556M 2023-12-14T18:21:40.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:40.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:40.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:40.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:21:40.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:40.574 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:40 smithi097 ceph-mon[43486]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 4 op/s 2023-12-14T18:21:40.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-14T18:21:40.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: osdmap e22: 4 total, 3 up, 4 in 2023-12-14T18:21:40.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:40.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:21:40.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi093 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi093 to 8556M 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:40.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.748 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:21:40.748 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:40.748 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:40 smithi160 ceph-mon[43840]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 4 op/s 2023-12-14T18:21:40.815 INFO:journalctl@ceph.osd.3.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-3[60361]: 2023-12-14T18:21:40.364+0000 7f097b0c9700 -1 osd.3 0 waiting for initial osdmap 2023-12-14T18:21:40.815 INFO:journalctl@ceph.osd.3.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-3[60361]: 2023-12-14T18:21:40.371+0000 7f09746b9700 -1 osd.3 23 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:21:40.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2023-12-14T18:21:40.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: osdmap e22: 4 total, 3 up, 4 in 2023-12-14T18:21:40.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:40.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:21:40.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi093 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi093 to 8556M 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:40.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:40.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:21:40.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:40.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:40 smithi093 ceph-mon[42070]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 17 MiB used, 268 GiB / 268 GiB avail; 1.2 KiB/s rd, 72 KiB/s wr, 4 op/s 2023-12-14T18:21:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.067+0000 7f2528c7a700 1 -- 172.21.15.97:0/1477035684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25240ff1e0 msgr2=0x7f25240ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.067+0000 7f2528c7a700 1 --2- 172.21.15.97:0/1477035684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25240ff1e0 0x7f25240ff5f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2514009990 tx=0x7f2514009ca0 comp rx=0 tx=0).stop 2023-12-14T18:21:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.067+0000 7f2528c7a700 1 -- 172.21.15.97:0/1477035684 shutdown_connections 2023-12-14T18:21:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.067+0000 7f2528c7a700 1 --2- 172.21.15.97:0/1477035684 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2524100d70 0x7f2524107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.067+0000 7f2528c7a700 1 --2- 172.21.15.97:0/1477035684 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25240ff1e0 0x7f25240ff5f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.067+0000 7f2528c7a700 1 --2- 172.21.15.97:0/1477035684 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25241003e0 0x7f2524100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.067+0000 7f2528c7a700 1 -- 172.21.15.97:0/1477035684 >> 172.21.15.97:0/1477035684 conn(0x7f25240fa7f0 msgr2=0x7f25240fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:41.068 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.067+0000 7f2528c7a700 1 -- 172.21.15.97:0/1477035684 shutdown_connections 2023-12-14T18:21:41.069 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.067+0000 7f2528c7a700 1 -- 172.21.15.97:0/1477035684 wait complete. 2023-12-14T18:21:41.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2528c7a700 1 Processor -- start 2023-12-14T18:21:41.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2528c7a700 1 -- start start 2023-12-14T18:21:41.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2528c7a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25240ff1e0 0x7f25240699a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:41.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2528c7a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25241003e0 0x7f252419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:41.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2528c7a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2524100d70 0x7f252419d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:41.071 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2528c7a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252419dbd0 con 0x7f25241003e0 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2528c7a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252419dd40 con 0x7f2524100d70 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2528c7a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f252419df00 con 0x7f25240ff1e0 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2522d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2524100d70 0x7f252419d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2522d9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2524100d70 0x7f252419d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:58084/0 (socket says 172.21.15.97:58084) 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f252259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25240ff1e0 0x7f25240699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2522d9d700 1 -- 172.21.15.97:0/315198991 learned_addr learned my addr 172.21.15.97:0/315198991 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2521d9b700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25241003e0 0x7f252419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2522d9d700 1 -- 172.21.15.97:0/315198991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25240ff1e0 msgr2=0x7f25240699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2522d9d700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25240ff1e0 0x7f25240699a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2522d9d700 1 -- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25241003e0 msgr2=0x7f252419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:41.072 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2522d9d700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25241003e0 0x7f252419adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2522d9d700 1 -- 172.21.15.97:0/315198991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25140095f0 con 0x7f2524100d70 2023-12-14T18:21:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2521d9b700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25241003e0 0x7f252419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:21:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.068+0000 7f2522d9d700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2524100d70 0x7f252419d430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f251c007ee0 tx=0x7f251c00d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.069+0000 7f25137fe700 1 -- 172.21.15.97:0/315198991 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c017070 con 0x7f2524100d70 2023-12-14T18:21:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.069+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f252410ab90 con 0x7f2524100d70 2023-12-14T18:21:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.069+0000 7f25137fe700 1 -- 172.21.15.97:0/315198991 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f251c00f040 con 0x7f2524100d70 2023-12-14T18:21:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.069+0000 7f25137fe700 1 -- 172.21.15.97:0/315198991 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f251c005740 con 0x7f2524100d70 2023-12-14T18:21:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.069+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f252410aa00 con 0x7f2524100d70 2023-12-14T18:21:41.073 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.070+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25241067f0 con 0x7f2524100d70 2023-12-14T18:21:41.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.071+0000 7f25137fe700 1 -- 172.21.15.97:0/315198991 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f251c00dc60 con 0x7f2524100d70 2023-12-14T18:21:41.074 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.071+0000 7f25137fe700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f250c0a1150 0x7f250c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.071+0000 7f25137fe700 1 -- 172.21.15.97:0/315198991 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 3702+0+0 (secure 0 0 0) 0x7f251c0caec0 con 0x7f2524100d70 2023-12-14T18:21:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.072+0000 7f252259c700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f250c0a1150 0x7f250c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:41.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.072+0000 7f252259c700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f250c0a1150 0x7f250c0a3600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f251400b5c0 tx=0x7f2514005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:41.076 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.076+0000 7f25137fe700 1 -- 172.21.15.97:0/315198991 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f251c095810 con 0x7f2524100d70 2023-12-14T18:21:41.273 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:41.272+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f2524061c70 con 0x7f250c0a1150 2023-12-14T18:21:41.720 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: osdmap e23: 4 total, 3 up, 4 in 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] boot 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: osdmap e24: 4 total, 4 up, 4 in 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:41.721 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:41.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:21:41.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: osdmap e23: 4 total, 3 up, 4 in 2023-12-14T18:21:41.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:41.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:41.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] boot 2023-12-14T18:21:41.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: osdmap e24: 4 total, 4 up, 4 in 2023-12-14T18:21:41.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:41.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:41.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:41.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:41.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:41.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: from='osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: osdmap e23: 4 total, 3 up, 4 in 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: osd.3 [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] boot 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: osdmap e24: 4 total, 4 up, 4 in 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: from='client.24194 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:41.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:42.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:42 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:21:42.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:42 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:21:42.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:42 smithi160 ceph-mon[43840]: osdmap e25: 4 total, 4 up, 4 in 2023-12-14T18:21:42.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:42 smithi160 ceph-mon[43840]: pgmap v78: 1 pgs: 1 remapped+peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:43.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:42 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:21:43.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:42 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:21:43.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:42 smithi093 ceph-mon[42070]: osdmap e25: 4 total, 4 up, 4 in 2023-12-14T18:21:43.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:42 smithi093 ceph-mon[42070]: pgmap v78: 1 pgs: 1 remapped+peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:43.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:42 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:21:43.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:42 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:21:43.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:42 smithi097 ceph-mon[43486]: osdmap e25: 4 total, 4 up, 4 in 2023-12-14T18:21:43.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:42 smithi097 ceph-mon[43486]: pgmap v78: 1 pgs: 1 remapped+peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:43.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:43 smithi160 ceph-mon[43840]: osdmap e26: 4 total, 4 up, 4 in 2023-12-14T18:21:43.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:43 smithi160 ceph-mon[43840]: from='client.? 172.21.15.97:0/3380291330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7afe6ea6-e5ce-4fea-8e55-a5fbcba54181"}]: dispatch 2023-12-14T18:21:43.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:43 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7afe6ea6-e5ce-4fea-8e55-a5fbcba54181"}]: dispatch 2023-12-14T18:21:43.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:43 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7afe6ea6-e5ce-4fea-8e55-a5fbcba54181"}]': finished 2023-12-14T18:21:43.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:43 smithi160 ceph-mon[43840]: osdmap e27: 5 total, 4 up, 5 in 2023-12-14T18:21:43.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:43 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:44.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:43 smithi093 ceph-mon[42070]: osdmap e26: 4 total, 4 up, 4 in 2023-12-14T18:21:44.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:43 smithi093 ceph-mon[42070]: from='client.? 172.21.15.97:0/3380291330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7afe6ea6-e5ce-4fea-8e55-a5fbcba54181"}]: dispatch 2023-12-14T18:21:44.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:43 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7afe6ea6-e5ce-4fea-8e55-a5fbcba54181"}]: dispatch 2023-12-14T18:21:44.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:43 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7afe6ea6-e5ce-4fea-8e55-a5fbcba54181"}]': finished 2023-12-14T18:21:44.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:43 smithi093 ceph-mon[42070]: osdmap e27: 5 total, 4 up, 5 in 2023-12-14T18:21:44.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:43 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:44.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:43 smithi097 ceph-mon[43486]: osdmap e26: 4 total, 4 up, 4 in 2023-12-14T18:21:44.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:43 smithi097 ceph-mon[43486]: from='client.? 172.21.15.97:0/3380291330' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7afe6ea6-e5ce-4fea-8e55-a5fbcba54181"}]: dispatch 2023-12-14T18:21:44.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:43 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7afe6ea6-e5ce-4fea-8e55-a5fbcba54181"}]: dispatch 2023-12-14T18:21:44.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:43 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7afe6ea6-e5ce-4fea-8e55-a5fbcba54181"}]': finished 2023-12-14T18:21:44.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:43 smithi097 ceph-mon[43486]: osdmap e27: 5 total, 4 up, 5 in 2023-12-14T18:21:44.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:43 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:44.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:44 smithi160 ceph-mon[43840]: from='client.? 172.21.15.97:0/566475033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:44.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:44 smithi160 ceph-mon[43840]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:44.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:44 smithi160 ceph-mon[43840]: osdmap e28: 5 total, 4 up, 5 in 2023-12-14T18:21:44.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:45.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:44 smithi093 ceph-mon[42070]: from='client.? 172.21.15.97:0/566475033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:45.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:44 smithi093 ceph-mon[42070]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:45.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:44 smithi093 ceph-mon[42070]: osdmap e28: 5 total, 4 up, 5 in 2023-12-14T18:21:45.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:45.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:44 smithi097 ceph-mon[43486]: from='client.? 172.21.15.97:0/566475033' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:21:45.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:44 smithi097 ceph-mon[43486]: pgmap v81: 1 pgs: 1 remapped+peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:45.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:44 smithi097 ceph-mon[43486]: osdmap e28: 5 total, 4 up, 5 in 2023-12-14T18:21:45.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:46 smithi160 ceph-mon[43840]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:46.997 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:46 smithi097 ceph-mon[43486]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:47.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:46 smithi093 ceph-mon[42070]: pgmap v83: 1 pgs: 1 remapped+peering; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:49.008 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:48 smithi097 ceph-mon[43486]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-14T18:21:49.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:48 smithi160 ceph-mon[43840]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-14T18:21:49.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:48 smithi093 ceph-mon[42070]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2023-12-14T18:21:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:49 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-14T18:21:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:49 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:49 smithi097 ceph-mon[43486]: Deploying daemon osd.4 on smithi097 2023-12-14T18:21:50.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:49 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-14T18:21:50.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:49 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:50.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:49 smithi160 ceph-mon[43840]: Deploying daemon osd.4 on smithi097 2023-12-14T18:21:50.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:49 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2023-12-14T18:21:50.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:49 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:50.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:49 smithi093 ceph-mon[42070]: Deploying daemon osd.4 on smithi097 2023-12-14T18:21:51.187 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:50 smithi097 ceph-mon[43486]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-12-14T18:21:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:50 smithi160 ceph-mon[43840]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-12-14T18:21:51.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:50 smithi093 ceph-mon[42070]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2023-12-14T18:21:53.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:52 smithi097 ceph-mon[43486]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-14T18:21:53.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:52 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:53.235 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:52 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:53.236 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:52 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:52 smithi160 ceph-mon[43840]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-14T18:21:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:52 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:52 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:52 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:53.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:52 smithi093 ceph-mon[42070]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-14T18:21:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:52 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:52 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:52 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.182 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.181+0000 7f25137fe700 1 -- 172.21.15.97:0/315198991 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f2524061c70 con 0x7f250c0a1150 2023-12-14T18:21:55.183 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 4 on host 'smithi097' 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f250c0a1150 msgr2=0x7f250c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f250c0a1150 0x7f250c0a3600 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f251400b5c0 tx=0x7f2514005fb0 comp rx=0 tx=0).stop 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2524100d70 msgr2=0x7f252419d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2524100d70 0x7f252419d430 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f251c007ee0 tx=0x7f251c00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 shutdown_connections 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f250c0a1150 0x7f250c0a3600 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2524100d70 0x7f252419d430 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25240ff1e0 0x7f25240699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 --2- 172.21.15.97:0/315198991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25241003e0 0x7f252419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 >> 172.21.15.97:0/315198991 conn(0x7f25240fa7f0 msgr2=0x7f2524103930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 shutdown_connections 2023-12-14T18:21:55.185 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:21:55.184+0000 7f2528c7a700 1 -- 172.21.15.97:0/315198991 wait complete. 2023-12-14T18:21:55.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:21:55.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:55 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:55 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:55.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:21:55.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:21:55.730 DEBUG:teuthology.orchestra.run.smithi097:osd.4> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.4.service 2023-12-14T18:21:55.732 INFO:tasks.cephadm:Deploying osd.5 on smithi160 with /dev/vg_nvme/lv_3... 2023-12-14T18:21:55.732 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2023-12-14T18:21:55.765 INFO:journalctl@ceph.osd.4.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:21:56.050 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:21:56.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:56 smithi160 ceph-mon[43840]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-14T18:21:57.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:56 smithi093 ceph-mon[42070]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-14T18:21:57.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:56 smithi097 ceph-mon[43486]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-14T18:21:57.093 INFO:journalctl@ceph.osd.4.smithi097.stdout:Dec 14 18:21:56 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-4[53874]: 2023-12-14T18:21:56.760+0000 7fcad9f32640 -1 osd.4 0 log_to_monitors true 2023-12-14T18:21:57.350 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_3 2023-12-14T18:21:57.350 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2023-12-14T18:21:57.350 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2023-12-14T18:21:57.350 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2023-12-14T18:21:57.350 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0116204 s, 902 MB/s 2023-12-14T18:21:57.350 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2023-12-14T18:21:57.748 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi160:vg_nvme/lv_3 2023-12-14T18:21:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='osd.4 [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:21:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:21:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:57.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:57.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:57.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:57 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:58.052 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:21:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='osd.4 [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:21:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:21:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:58.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:58.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:57 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:58.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='osd.4 [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:21:58.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2023-12-14T18:21:58.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:58.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:58.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:58.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:21:58.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:58.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:21:58.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:21:58.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:57 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:21:58.942 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.938+0000 7fd8016b1700 1 -- 172.21.15.160:0/3521554203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101510 msgr2=0x7fd7fc101960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:58.942 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.938+0000 7fd8016b1700 1 --2- 172.21.15.160:0/3521554203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101510 0x7fd7fc101960 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd7ec009990 tx=0x7fd7ec009ca0 comp rx=0 tx=0).stop 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.938+0000 7fd8016b1700 1 -- 172.21.15.160:0/3521554203 shutdown_connections 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.938+0000 7fd8016b1700 1 --2- 172.21.15.160:0/3521554203 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7fc100310 0x7fd7fc100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.938+0000 7fd8016b1700 1 --2- 172.21.15.160:0/3521554203 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7fc101ea0 0x7fd7fc108760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.938+0000 7fd8016b1700 1 --2- 172.21.15.160:0/3521554203 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101510 0x7fd7fc101960 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.938+0000 7fd8016b1700 1 -- 172.21.15.160:0/3521554203 >> 172.21.15.160:0/3521554203 conn(0x7fd7fc0fb960 msgr2=0x7fd7fc0fdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.938+0000 7fd8016b1700 1 -- 172.21.15.160:0/3521554203 shutdown_connections 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.938+0000 7fd8016b1700 1 -- 172.21.15.160:0/3521554203 wait complete. 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd8016b1700 1 Processor -- start 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd8016b1700 1 -- start start 2023-12-14T18:21:58.943 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd8016b1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7fc100310 0x7fd7fc196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd8016b1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7fc101510 0x7fd7fc196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd8016b1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101ea0 0x7fd7fc19d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd8016b1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc19dbb0 con 0x7fd7fc100310 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd8016b1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc19dcf0 con 0x7fd7fc101510 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd8016b1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7fc19de80 con 0x7fd7fc101ea0 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7fb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101ea0 0x7fd7fc19d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7fb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101ea0 0x7fd7fc19d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:42240/0 (socket says 172.21.15.160:42240) 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7fb7fe700 1 -- 172.21.15.160:0/1827527093 learned_addr learned my addr 172.21.15.160:0/1827527093 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7fa7fc700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7fc101510 0x7fd7fc196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7faffd700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7fc100310 0x7fd7fc196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:58.944 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7fb7fe700 1 -- 172.21.15.160:0/1827527093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7fc101510 msgr2=0x7fd7fc196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:58.945 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7fb7fe700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7fc101510 0x7fd7fc196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:58.945 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7fb7fe700 1 -- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7fc100310 msgr2=0x7fd7fc196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:21:58.945 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7fb7fe700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7fc100310 0x7fd7fc196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:21:58.945 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7fb7fe700 1 -- 172.21.15.160:0/1827527093 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7ec0095f0 con 0x7fd7fc101ea0 2023-12-14T18:21:58.945 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.939+0000 7fd7faffd700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7fc100310 0x7fd7fc196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:21:58.945 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.940+0000 7fd7fb7fe700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101ea0 0x7fd7fc19d5e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4007f10 tx=0x7fd7f400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:58.945 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.940+0000 7fd7e3fff700 1 -- 172.21.15.160:0/1827527093 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4017070 con 0x7fd7fc101ea0 2023-12-14T18:21:58.945 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.940+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7fc10bc70 con 0x7fd7fc101ea0 2023-12-14T18:21:58.945 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.940+0000 7fd7e3fff700 1 -- 172.21.15.160:0/1827527093 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd7f400f040 con 0x7fd7fc101ea0 2023-12-14T18:21:58.946 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.940+0000 7fd7e3fff700 1 -- 172.21.15.160:0/1827527093 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f4005740 con 0x7fd7fc101ea0 2023-12-14T18:21:58.946 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.940+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7fc10bae0 con 0x7fd7fc101ea0 2023-12-14T18:21:58.946 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.941+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7fc1078b0 con 0x7fd7fc101ea0 2023-12-14T18:21:58.947 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.942+0000 7fd7e3fff700 1 -- 172.21.15.160:0/1827527093 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd7f40058a0 con 0x7fd7fc101ea0 2023-12-14T18:21:58.947 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.942+0000 7fd7e3fff700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7e40a1150 0x7fd7e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:21:58.947 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.943+0000 7fd7e3fff700 1 -- 172.21.15.160:0/1827527093 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 3821+0+0 (secure 0 0 0) 0x7fd7f40cb050 con 0x7fd7fc101ea0 2023-12-14T18:21:58.947 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.943+0000 7fd7faffd700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7e40a1150 0x7fd7e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:21:58.947 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.943+0000 7fd7faffd700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7e40a1150 0x7fd7e40a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc101370 tx=0x7fd7f0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:21:58.948 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:58.947+0000 7fd7e3fff700 1 -- 172.21.15.160:0/1827527093 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd7f40fd730 con 0x7fd7fc101ea0 2023-12-14T18:21:58.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi097 2023-12-14T18:21:58.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi097 to 8557M 2023-12-14T18:21:58.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:58.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:21:58.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:21:58.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-14T18:21:58.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: osdmap e29: 5 total, 4 up, 5 in 2023-12-14T18:21:58.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:58.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: from='osd.4 [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:58.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:58.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:58 smithi160 ceph-mon[43840]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:59.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi097 2023-12-14T18:21:59.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi097 to 8557M 2023-12-14T18:21:59.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:59.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:21:59.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:21:59.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-14T18:21:59.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: osdmap e29: 5 total, 4 up, 5 in 2023-12-14T18:21:59.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:59.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: from='osd.4 [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:59.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:59.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:58 smithi093 ceph-mon[42070]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:59.092 INFO:journalctl@ceph.osd.4.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-4[53874]: 2023-12-14T18:21:58.732+0000 7fcacf5a9700 -1 osd.4 0 waiting for initial osdmap 2023-12-14T18:21:59.093 INFO:journalctl@ceph.osd.4.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-4[53874]: 2023-12-14T18:21:58.740+0000 7fcac939a700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:21:59.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi097 2023-12-14T18:21:59.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi097 to 8557M 2023-12-14T18:21:59.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:59.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:21:59.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:21:59.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2023-12-14T18:21:59.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: osdmap e29: 5 total, 4 up, 5 in 2023-12-14T18:21:59.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:59.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: from='osd.4 [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:59.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:21:59.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:58 smithi097 ceph-mon[43486]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 22 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:21:59.145 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:21:59.144+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7fc0619e0 con 0x7fd7e40a1150 2023-12-14T18:21:59.869 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:59 smithi160 ceph-mon[43840]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:21:59.869 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:59 smithi160 ceph-mon[43840]: osdmap e30: 5 total, 4 up, 5 in 2023-12-14T18:21:59.869 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:59.869 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:21:59.869 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:59 smithi160 ceph-mon[43840]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:21:59.869 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:21:59.869 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:21:59.870 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:21:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:00.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:59 smithi093 ceph-mon[42070]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:22:00.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:59 smithi093 ceph-mon[42070]: osdmap e30: 5 total, 4 up, 5 in 2023-12-14T18:22:00.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:00.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:00.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:59 smithi093 ceph-mon[42070]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:00.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:00.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:00.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:21:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:00.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:59 smithi097 ceph-mon[43486]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:22:00.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:59 smithi097 ceph-mon[43486]: osdmap e30: 5 total, 4 up, 5 in 2023-12-14T18:22:00.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:00.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:00.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:59 smithi097 ceph-mon[43486]: from='client.24220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:00.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:00.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:00.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:21:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:00.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:00 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:22:00.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:00 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:22:00.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:00 smithi160 ceph-mon[43840]: osd.4 [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] boot 2023-12-14T18:22:00.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:00 smithi160 ceph-mon[43840]: osdmap e31: 5 total, 5 up, 5 in 2023-12-14T18:22:00.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:00 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:00.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:00 smithi160 ceph-mon[43840]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:22:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:00 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:22:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:00 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:22:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:00 smithi093 ceph-mon[42070]: osd.4 [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] boot 2023-12-14T18:22:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:00 smithi093 ceph-mon[42070]: osdmap e31: 5 total, 5 up, 5 in 2023-12-14T18:22:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:00 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:00 smithi093 ceph-mon[42070]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:22:01.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:00 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:22:01.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:00 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:22:01.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:00 smithi097 ceph-mon[43486]: osd.4 [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] boot 2023-12-14T18:22:01.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:00 smithi097 ceph-mon[43486]: osdmap e31: 5 total, 5 up, 5 in 2023-12-14T18:22:01.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:00 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:22:01.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:00 smithi097 ceph-mon[43486]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 23 MiB used, 358 GiB / 358 GiB avail 2023-12-14T18:22:01.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:01 smithi160 ceph-mon[43840]: osdmap e32: 5 total, 5 up, 5 in 2023-12-14T18:22:01.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:01 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/2286752102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6882d38d-40ad-40d7-aeb3-8c1e9367ef16"}]: dispatch 2023-12-14T18:22:01.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:01 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6882d38d-40ad-40d7-aeb3-8c1e9367ef16"}]: dispatch 2023-12-14T18:22:01.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:01 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6882d38d-40ad-40d7-aeb3-8c1e9367ef16"}]': finished 2023-12-14T18:22:01.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:01 smithi160 ceph-mon[43840]: osdmap e33: 6 total, 5 up, 6 in 2023-12-14T18:22:01.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:01 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:01.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:01 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/1952598965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:01 smithi093 ceph-mon[42070]: osdmap e32: 5 total, 5 up, 5 in 2023-12-14T18:22:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:01 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/2286752102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6882d38d-40ad-40d7-aeb3-8c1e9367ef16"}]: dispatch 2023-12-14T18:22:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:01 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6882d38d-40ad-40d7-aeb3-8c1e9367ef16"}]: dispatch 2023-12-14T18:22:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:01 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6882d38d-40ad-40d7-aeb3-8c1e9367ef16"}]': finished 2023-12-14T18:22:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:01 smithi093 ceph-mon[42070]: osdmap e33: 6 total, 5 up, 6 in 2023-12-14T18:22:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:01 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:01 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/1952598965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:02.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:01 smithi097 ceph-mon[43486]: osdmap e32: 5 total, 5 up, 5 in 2023-12-14T18:22:02.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:01 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/2286752102' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6882d38d-40ad-40d7-aeb3-8c1e9367ef16"}]: dispatch 2023-12-14T18:22:02.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:01 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6882d38d-40ad-40d7-aeb3-8c1e9367ef16"}]: dispatch 2023-12-14T18:22:02.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:01 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6882d38d-40ad-40d7-aeb3-8c1e9367ef16"}]': finished 2023-12-14T18:22:02.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:01 smithi097 ceph-mon[43486]: osdmap e33: 6 total, 5 up, 6 in 2023-12-14T18:22:02.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:01 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:02.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:01 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/1952598965' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:03.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:02 smithi093 ceph-mon[42070]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:03.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:02 smithi097 ceph-mon[43486]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:03.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:02 smithi160 ceph-mon[43840]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:05.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:04 smithi160 ceph-mon[43840]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:05.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:04 smithi093 ceph-mon[42070]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:05.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:04 smithi097 ceph-mon[43486]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:06.880 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:06 smithi160 ceph-mon[43840]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:07.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:06 smithi093 ceph-mon[42070]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:07.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:06 smithi097 ceph-mon[43486]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:07.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:07 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-14T18:22:07.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:07 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:07.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:07 smithi160 ceph-mon[43840]: Deploying daemon osd.5 on smithi160 2023-12-14T18:22:08.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:07 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-14T18:22:08.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:07 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:08.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:07 smithi093 ceph-mon[42070]: Deploying daemon osd.5 on smithi160 2023-12-14T18:22:08.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:07 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2023-12-14T18:22:08.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:07 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:08.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:07 smithi097 ceph-mon[43486]: Deploying daemon osd.5 on smithi160 2023-12-14T18:22:08.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:08 smithi160 ceph-mon[43840]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:09.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:08 smithi093 ceph-mon[42070]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:09.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:08 smithi097 ceph-mon[43486]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:11.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:10 smithi160 ceph-mon[43840]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:11.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:11.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:11.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:11.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:10 smithi093 ceph-mon[42070]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:11.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:11.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:11.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:11.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:10 smithi097 ceph-mon[43486]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:11.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:11.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:11.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:12.696 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 5 on host 'smithi160' 2023-12-14T18:22:12.697 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.695+0000 7fd7e3fff700 1 -- 172.21.15.160:0/1827527093 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fd7fc0619e0 con 0x7fd7e40a1150 2023-12-14T18:22:12.698 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7e40a1150 msgr2=0x7fd7e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7e40a1150 0x7fd7e40a3600 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd7fc101370 tx=0x7fd7f0005f90 comp rx=0 tx=0).stop 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101ea0 msgr2=0x7fd7fc19d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101ea0 0x7fd7fc19d5e0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd7f4007f10 tx=0x7fd7f400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 shutdown_connections 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7e40a1150 0x7fd7e40a3600 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd7fc100310 0x7fd7fc196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd7fc101ea0 0x7fd7fc19d5e0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 --2- 172.21.15.160:0/1827527093 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd7fc101510 0x7fd7fc196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 >> 172.21.15.160:0/1827527093 conn(0x7fd7fc0fb960 msgr2=0x7fd7fc1049f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 shutdown_connections 2023-12-14T18:22:12.699 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:12.697+0000 7fd8016b1700 1 -- 172.21.15.160:0/1827527093 wait complete. 2023-12-14T18:22:13.068 DEBUG:teuthology.orchestra.run.smithi160:osd.5> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.5.service 2023-12-14T18:22:13.071 INFO:tasks.cephadm:Deploying osd.6 on smithi093 with /dev/vg_nvme/lv_2... 2023-12-14T18:22:13.071 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-14T18:22:13.105 INFO:journalctl@ceph.osd.5.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:22:13.282 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:13 smithi160 ceph-mon[43840]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:13.282 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.283 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.283 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:13.283 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:13.283 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.283 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.283 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.283 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:13 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:13.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:13 smithi097 ceph-mon[43486]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:13 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:13.382 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:22:13.488 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:13 smithi093 ceph-mon[42070]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:13.489 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.489 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.489 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:13.489 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:13.489 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.489 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.489 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:13.489 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:13 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:14.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:14 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:14.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:14 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:22:14.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:14 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:14.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:14 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:14.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:14 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:22:14.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:14 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:14.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:14 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:14.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:14 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:22:14.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:14 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:14.595 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-14T18:22:14.595 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-14T18:22:14.595 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-12-14T18:22:14.595 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-12-14T18:22:14.595 INFO:teuthology.orchestra.run.smithi093.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113877 s, 921 MB/s 2023-12-14T18:22:14.595 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-12-14T18:22:14.795 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:22:14 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:22:14.523+0000 7f958d516640 -1 osd.5 0 log_to_monitors true 2023-12-14T18:22:14.985 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi093:vg_nvme/lv_2 2023-12-14T18:22:15.294 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='osd.5 [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='osd.5 [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:15.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='osd.5 [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:22:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2023-12-14T18:22:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:15.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:16.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.217+0000 7f7887c34700 1 -- 172.21.15.93:0/4294287705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880100d70 msgr2=0x7f7880103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:16.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.217+0000 7f7887c34700 1 --2- 172.21.15.93:0/4294287705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880100d70 0x7f7880103150 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f787c009990 tx=0x7f787c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:22:16.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.217+0000 7f7887c34700 1 -- 172.21.15.93:0/4294287705 shutdown_connections 2023-12-14T18:22:16.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.217+0000 7f7887c34700 1 --2- 172.21.15.93:0/4294287705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880100d70 0x7f7880103150 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:16.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.217+0000 7f7887c34700 1 --2- 172.21.15.93:0/4294287705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880105fb0 0x7f7880108410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:16.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.217+0000 7f7887c34700 1 --2- 172.21.15.93:0/4294287705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7880103690 0x7f7880105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:16.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.217+0000 7f7887c34700 1 -- 172.21.15.93:0/4294287705 >> 172.21.15.93:0/4294287705 conn(0x7f78800fa7f0 msgr2=0x7f78800fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:16.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.217+0000 7f7887c34700 1 -- 172.21.15.93:0/4294287705 shutdown_connections 2023-12-14T18:22:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.217+0000 7f7887c34700 1 -- 172.21.15.93:0/4294287705 wait complete. 2023-12-14T18:22:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f7887c34700 1 Processor -- start 2023-12-14T18:22:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f7887c34700 1 -- start start 2023-12-14T18:22:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f7887c34700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7880100d70 0x7f7880196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f7887c34700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880103690 0x7f7880196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f7887c34700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880105fb0 0x7f788019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f7887c34700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788019dbd0 con 0x7f7880103690 2023-12-14T18:22:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f7887c34700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788019dd10 con 0x7f7880105fb0 2023-12-14T18:22:16.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f7887c34700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788019dea0 con 0x7f7880100d70 2023-12-14T18:22:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78851cf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880103690 0x7f7880196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78851cf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880103690 0x7f7880196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37644/0 (socket says 172.21.15.93:37644) 2023-12-14T18:22:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78851cf700 1 -- 172.21.15.93:0/1386109524 learned_addr learned my addr 172.21.15.93:0/1386109524 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:22:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78859d0700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7880100d70 0x7f7880196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78851cf700 1 -- 172.21.15.93:0/1386109524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7880100d70 msgr2=0x7f7880196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78851cf700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7880100d70 0x7f7880196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78861d1700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880105fb0 0x7f788019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78851cf700 1 -- 172.21.15.93:0/1386109524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880105fb0 msgr2=0x7f788019d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:16.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78851cf700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880105fb0 0x7f788019d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:16.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78851cf700 1 -- 172.21.15.93:0/1386109524 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f787c0095f0 con 0x7f7880103690 2023-12-14T18:22:16.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78861d1700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880105fb0 0x7f788019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:22:16.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.218+0000 7f78851cf700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880103690 0x7f7880196580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f787000c950 tx=0x7f787000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:16.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.219+0000 7f7876ffd700 1 -- 172.21.15.93:0/1386109524 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f787000cea0 con 0x7f7880103690 2023-12-14T18:22:16.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.219+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788010bcd0 con 0x7f7880103690 2023-12-14T18:22:16.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.219+0000 7f7876ffd700 1 -- 172.21.15.93:0/1386109524 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7870004830 con 0x7f7880103690 2023-12-14T18:22:16.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.219+0000 7f7876ffd700 1 -- 172.21.15.93:0/1386109524 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7870005460 con 0x7f7880103690 2023-12-14T18:22:16.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.219+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788010bb40 con 0x7f7880103690 2023-12-14T18:22:16.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.220+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78800fc360 con 0x7f7880103690 2023-12-14T18:22:16.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.221+0000 7f7876ffd700 1 -- 172.21.15.93:0/1386109524 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7870007ac0 con 0x7f7880103690 2023-12-14T18:22:16.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.222+0000 7f7876ffd700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f786c0a1150 0x7f786c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:16.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.222+0000 7f78859d0700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f786c0a1150 0x7f786c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:16.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.222+0000 7f7876ffd700 1 -- 172.21.15.93:0/1386109524 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 4253+0+0 (secure 0 0 0) 0x7f78700cb980 con 0x7f7880103690 2023-12-14T18:22:16.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.222+0000 7f78859d0700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f786c0a1150 0x7f786c0a3600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f787c00b5c0 tx=0x7f787c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:16.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.226+0000 7f7876ffd700 1 -- 172.21.15.93:0/1386109524 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f78700960b0 con 0x7f7880103690 2023-12-14T18:22:16.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi160 2023-12-14T18:22:16.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi160 to 8557M 2023-12-14T18:22:16.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:22:16.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-14T18:22:16.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: osdmap e34: 6 total, 5 up, 6 in 2023-12-14T18:22:16.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:16.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: from='osd.5 [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:22:16.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:16 smithi093 ceph-mon[42070]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:22:16.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:16.425+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f7880061be0 con 0x7f786c0a1150 2023-12-14T18:22:16.496 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:22:16.108+0000 7f9582b8d700 -1 osd.5 0 waiting for initial osdmap 2023-12-14T18:22:16.496 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:22:16.116+0000 7f957c97e700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:22:16.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi160 2023-12-14T18:22:16.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi160 to 8557M 2023-12-14T18:22:16.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:22:16.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-14T18:22:16.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: osdmap e34: 6 total, 5 up, 6 in 2023-12-14T18:22:16.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:16.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: from='osd.5 [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:22:16.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:16 smithi160 ceph-mon[43840]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi160 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi160 to 8557M 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: osdmap e34: 6 total, 5 up, 6 in 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: from='osd.5 [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:22:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:16 smithi097 ceph-mon[43486]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:22:17.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:17.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:22:17.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: osdmap e35: 6 total, 5 up, 6 in 2023-12-14T18:22:17.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:17.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:17.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:17.147 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:17.148 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:17.148 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: osd.5 [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] boot 2023-12-14T18:22:17.148 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: osdmap e36: 6 total, 6 up, 6 in 2023-12-14T18:22:17.148 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: osdmap e35: 6 total, 5 up, 6 in 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: osd.5 [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] boot 2023-12-14T18:22:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: osdmap e36: 6 total, 6 up, 6 in 2023-12-14T18:22:17.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:17.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 28 MiB used, 447 GiB / 447 GiB avail 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: osdmap e35: 6 total, 5 up, 6 in 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: osd.5 [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] boot 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: osdmap e36: 6 total, 6 up, 6 in 2023-12-14T18:22:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:22:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:18 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:22:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:18 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:22:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:18 smithi160 ceph-mon[43840]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:18 smithi160 ceph-mon[43840]: osdmap e37: 6 total, 6 up, 6 in 2023-12-14T18:22:18.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:18 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:22:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:18 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:22:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:18 smithi093 ceph-mon[42070]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:18 smithi093 ceph-mon[42070]: osdmap e37: 6 total, 6 up, 6 in 2023-12-14T18:22:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:18 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:22:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:18 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:22:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:18 smithi097 ceph-mon[43486]: from='client.14370 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:18.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:18 smithi097 ceph-mon[43486]: osdmap e37: 6 total, 6 up, 6 in 2023-12-14T18:22:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:19 smithi160 ceph-mon[43840]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:19 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3110878921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f046b70-3c19-4dac-99f8-2c72520268be"}]: dispatch 2023-12-14T18:22:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:19 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3110878921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f046b70-3c19-4dac-99f8-2c72520268be"}]': finished 2023-12-14T18:22:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:19 smithi160 ceph-mon[43840]: osdmap e38: 7 total, 6 up, 7 in 2023-12-14T18:22:19.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:19.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:19 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3777646958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:19 smithi093 ceph-mon[42070]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:19 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3110878921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f046b70-3c19-4dac-99f8-2c72520268be"}]: dispatch 2023-12-14T18:22:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:19 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3110878921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f046b70-3c19-4dac-99f8-2c72520268be"}]': finished 2023-12-14T18:22:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:19 smithi093 ceph-mon[42070]: osdmap e38: 7 total, 6 up, 7 in 2023-12-14T18:22:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:19 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3777646958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:19.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:19 smithi097 ceph-mon[43486]: pgmap v107: 1 pgs: 1 peering; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:19 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3110878921' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4f046b70-3c19-4dac-99f8-2c72520268be"}]: dispatch 2023-12-14T18:22:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:19 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3110878921' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4f046b70-3c19-4dac-99f8-2c72520268be"}]': finished 2023-12-14T18:22:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:19 smithi097 ceph-mon[43486]: osdmap e38: 7 total, 6 up, 7 in 2023-12-14T18:22:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:19 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3777646958' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:20.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:20 smithi160 ceph-mon[43840]: osdmap e39: 7 total, 6 up, 7 in 2023-12-14T18:22:20.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:20.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:20 smithi093 ceph-mon[42070]: osdmap e39: 7 total, 6 up, 7 in 2023-12-14T18:22:20.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:20.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:20 smithi097 ceph-mon[43486]: osdmap e39: 7 total, 6 up, 7 in 2023-12-14T18:22:20.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:21.495 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:21 smithi160 ceph-mon[43840]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:21.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:21 smithi160 ceph-mon[43840]: osdmap e40: 7 total, 6 up, 7 in 2023-12-14T18:22:21.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:21.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:21 smithi093 ceph-mon[42070]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:21.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:21 smithi093 ceph-mon[42070]: osdmap e40: 7 total, 6 up, 7 in 2023-12-14T18:22:21.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:21.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:21 smithi097 ceph-mon[43486]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 33 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:21.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:21 smithi097 ceph-mon[43486]: osdmap e40: 7 total, 6 up, 7 in 2023-12-14T18:22:21.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:23 smithi160 ceph-mon[43840]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:22:23.521 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:23 smithi093 ceph-mon[42070]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:22:23.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:23 smithi097 ceph-mon[43486]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:22:25.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:25 smithi160 ceph-mon[43840]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-12-14T18:22:25.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-14T18:22:25.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:25.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:25 smithi160 ceph-mon[43840]: Deploying daemon osd.6 on smithi093 2023-12-14T18:22:25.504 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:25 smithi093 ceph-mon[42070]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-12-14T18:22:25.505 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-14T18:22:25.505 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:25.505 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:25 smithi093 ceph-mon[42070]: Deploying daemon osd.6 on smithi093 2023-12-14T18:22:25.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:25 smithi097 ceph-mon[43486]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 99 KiB/s, 0 objects/s recovering 2023-12-14T18:22:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2023-12-14T18:22:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:25 smithi097 ceph-mon[43486]: Deploying daemon osd.6 on smithi093 2023-12-14T18:22:26.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:26 smithi160 ceph-mon[43840]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-14T18:22:27.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:26 smithi093 ceph-mon[42070]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-14T18:22:27.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:26 smithi097 ceph-mon[43486]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 75 KiB/s, 0 objects/s recovering 2023-12-14T18:22:29.240 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:29 smithi093 ceph-mon[42070]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-14T18:22:29.241 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:29 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:29.241 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:29 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:29.241 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:29 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:29.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:29 smithi097 ceph-mon[43486]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-14T18:22:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:29 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:29 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:29 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:29.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:29 smithi160 ceph-mon[43840]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 66 KiB/s, 0 objects/s recovering 2023-12-14T18:22:29.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:29 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:29.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:29 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:29.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:29 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:30.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.167+0000 7f7876ffd700 1 -- 172.21.15.93:0/1386109524 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f7880061be0 con 0x7f786c0a1150 2023-12-14T18:22:30.169 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 6 on host 'smithi093' 2023-12-14T18:22:30.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f786c0a1150 msgr2=0x7f786c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:30.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f786c0a1150 0x7f786c0a3600 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f787c00b5c0 tx=0x7f787c005fd0 comp rx=0 tx=0).stop 2023-12-14T18:22:30.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880103690 msgr2=0x7f7880196580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:30.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880103690 0x7f7880196580 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f787000c950 tx=0x7f787000cc60 comp rx=0 tx=0).stop 2023-12-14T18:22:30.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 shutdown_connections 2023-12-14T18:22:30.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f786c0a1150 0x7f786c0a3600 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:30.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7880105fb0 0x7f788019d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:30.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7880100d70 0x7f7880196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:30.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 --2- 172.21.15.93:0/1386109524 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7880103690 0x7f7880196580 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:30.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.169+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 >> 172.21.15.93:0/1386109524 conn(0x7f78800fa7f0 msgr2=0x7f78801019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:30.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.170+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 shutdown_connections 2023-12-14T18:22:30.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:22:30.170+0000 7f7887c34700 1 -- 172.21.15.93:0/1386109524 wait complete. 2023-12-14T18:22:30.606 DEBUG:teuthology.orchestra.run.smithi093:osd.6> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.6.service 2023-12-14T18:22:30.608 INFO:tasks.cephadm:Deploying osd.7 on smithi097 with /dev/vg_nvme/lv_2... 2023-12-14T18:22:30.609 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-14T18:22:30.640 INFO:journalctl@ceph.osd.6.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:22:30.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:30.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:30.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:30.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:22:30.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:30.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:22:30.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:30.925 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:22:30.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:30.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:30.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:30.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:22:30.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:30.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:22:30.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:30.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:31.039 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:31.040 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:31.041 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:32.157 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:22:32.019+0000 7fa7dbc22640 -1 osd.6 0 log_to_monitors true 2023-12-14T18:22:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-14T18:22:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-14T18:22:32.210 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-12-14T18:22:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-12-14T18:22:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0135556 s, 774 MB/s 2023-12-14T18:22:32.210 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-12-14T18:22:32.639 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_2 2023-12-14T18:22:32.943 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:22:33.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-14T18:22:33.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-14T18:22:33.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi093 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi093 to 5704M 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.248 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.248 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:22:33.248 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:32 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-14T18:22:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-14T18:22:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi093 2023-12-14T18:22:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi093 to 5704M 2023-12-14T18:22:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:22:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:32 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi093 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi093 to 5704M 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:33.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:33.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:33.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:22:33.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:32 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:33.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.966+0000 7f32086dd700 1 -- 172.21.15.97:0/2725486546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 msgr2=0x7f3200108770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:33.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.966+0000 7f32086dd700 1 --2- 172.21.15.97:0/2725486546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 0x7f3200108770 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f31f800b0a0 tx=0x7f31f800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:22:33.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.966+0000 7f32086dd700 1 -- 172.21.15.97:0/2725486546 shutdown_connections 2023-12-14T18:22:33.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.966+0000 7f32086dd700 1 --2- 172.21.15.97:0/2725486546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 0x7f3200108770 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:33.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.966+0000 7f32086dd700 1 --2- 172.21.15.97:0/2725486546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3200100320 0x7f3200100730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:33.971 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.966+0000 7f32086dd700 1 --2- 172.21.15.97:0/2725486546 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3200101520 0x7f3200101970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.966+0000 7f32086dd700 1 -- 172.21.15.97:0/2725486546 >> 172.21.15.97:0/2725486546 conn(0x7f32000fb970 msgr2=0x7f32000fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.966+0000 7f32086dd700 1 -- 172.21.15.97:0/2725486546 shutdown_connections 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.966+0000 7f32086dd700 1 -- 172.21.15.97:0/2725486546 wait complete. 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f32086dd700 1 Processor -- start 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f32086dd700 1 -- start start 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f32086dd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3200100320 0x7f3200074890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f32086dd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3200101520 0x7f3200072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f32086dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 0x7f3200073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f32086dd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200073ba0 con 0x7f3200100320 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f32086dd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3200073d10 con 0x7f3200101eb0 2023-12-14T18:22:33.972 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f32086dd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32001a0190 con 0x7f3200101520 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f3206c7a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 0x7f3200073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f3206c7a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 0x7f3200073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:45170/0 (socket says 172.21.15.97:45170) 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f3206c7a700 1 -- 172.21.15.97:0/991584015 learned_addr learned my addr 172.21.15.97:0/991584015 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f3206479700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3200100320 0x7f3200074890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.967+0000 7f3205c78700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3200101520 0x7f3200072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f3206c7a700 1 -- 172.21.15.97:0/991584015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3200101520 msgr2=0x7f3200072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f3206c7a700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3200101520 0x7f3200072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f3206c7a700 1 -- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3200100320 msgr2=0x7f3200074890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f3206c7a700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3200100320 0x7f3200074890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f3206c7a700 1 -- 172.21.15.97:0/991584015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f31f8009d00 con 0x7f3200101eb0 2023-12-14T18:22:33.973 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f3205c78700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3200101520 0x7f3200072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:22:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f3206c7a700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 0x7f3200073420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f31f80062a0 tx=0x7f31f8004d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f31f37fe700 1 -- 172.21.15.97:0/991584015 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f800e040 con 0x7f3200101eb0 2023-12-14T18:22:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f31f37fe700 1 -- 172.21.15.97:0/991584015 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f31f800cec0 con 0x7f3200101eb0 2023-12-14T18:22:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f320010bc90 con 0x7f3200101eb0 2023-12-14T18:22:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f31f37fe700 1 -- 172.21.15.97:0/991584015 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f31f8004020 con 0x7f3200101eb0 2023-12-14T18:22:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.968+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f320010bb00 con 0x7f3200101eb0 2023-12-14T18:22:33.974 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.969+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3200107870 con 0x7f3200101eb0 2023-12-14T18:22:33.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.970+0000 7f31f37fe700 1 -- 172.21.15.97:0/991584015 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f31f801d690 con 0x7f3200101eb0 2023-12-14T18:22:33.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.970+0000 7f31f37fe700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f31ec0a11d0 0x7f31ec0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:33.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.971+0000 7f31f37fe700 1 -- 172.21.15.97:0/991584015 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 4669+0+0 (secure 0 0 0) 0x7f31f80cd920 con 0x7f3200101eb0 2023-12-14T18:22:33.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.971+0000 7f3206479700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f31ec0a11d0 0x7f31ec0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:33.975 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.971+0000 7f3206479700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f31ec0a11d0 0x7f31ec0a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3200101380 tx=0x7f31f400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:33.976 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:33.974+0000 7f31f37fe700 1 -- 172.21.15.97:0/991584015 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f31f8097f40 con 0x7f3200101eb0 2023-12-14T18:22:34.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:33 smithi097 ceph-mon[43486]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-14T18:22:34.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:33 smithi097 ceph-mon[43486]: osdmap e41: 7 total, 6 up, 7 in 2023-12-14T18:22:34.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:34.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:33 smithi097 ceph-mon[43486]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:22:34.173 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:34.171+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f3200061a70 con 0x7f31ec0a11d0 2023-12-14T18:22:34.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:33 smithi160 ceph-mon[43840]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-14T18:22:34.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:33 smithi160 ceph-mon[43840]: osdmap e41: 7 total, 6 up, 7 in 2023-12-14T18:22:34.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:34.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:33 smithi160 ceph-mon[43840]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:22:34.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:22:33 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:22:33.985+0000 7fa7d1299700 -1 osd.6 0 waiting for initial osdmap 2023-12-14T18:22:34.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:22:33.998+0000 7fa7cd08e700 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:22:34.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:33 smithi093 ceph-mon[42070]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2023-12-14T18:22:34.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:33 smithi093 ceph-mon[42070]: osdmap e41: 7 total, 6 up, 7 in 2023-12-14T18:22:34.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:34.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:33 smithi093 ceph-mon[42070]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:22:35.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:34 smithi160 ceph-mon[43840]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:35.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:34 smithi160 ceph-mon[43840]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:22:35.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:34 smithi160 ceph-mon[43840]: osdmap e42: 7 total, 6 up, 7 in 2023-12-14T18:22:35.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:34 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:35.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:34 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:35.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:34 smithi160 ceph-mon[43840]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:35.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:34 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:35.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:34 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:35.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:34 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:35.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:34 smithi097 ceph-mon[43486]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:35.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:34 smithi097 ceph-mon[43486]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:22:35.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:34 smithi097 ceph-mon[43486]: osdmap e42: 7 total, 6 up, 7 in 2023-12-14T18:22:35.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:34 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:35.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:34 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:35.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:34 smithi097 ceph-mon[43486]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:35.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:34 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:35.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:34 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:35.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:34 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:35.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-mon[42070]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:35.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-mon[42070]: from='osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:22:35.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-mon[42070]: osdmap e42: 7 total, 6 up, 7 in 2023-12-14T18:22:35.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:35.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:35.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-mon[42070]: from='client.24263 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:35.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:35.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:35.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:34 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:36.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:35 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:22:36.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:35 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:22:36.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:35 smithi160 ceph-mon[43840]: osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] boot 2023-12-14T18:22:36.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:35 smithi160 ceph-mon[43840]: osdmap e43: 7 total, 7 up, 7 in 2023-12-14T18:22:36.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:36.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:35 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:22:36.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:35 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:22:36.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:35 smithi093 ceph-mon[42070]: osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] boot 2023-12-14T18:22:36.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:35 smithi093 ceph-mon[42070]: osdmap e43: 7 total, 7 up, 7 in 2023-12-14T18:22:36.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:36.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:35 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:22:36.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:35 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:22:36.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:35 smithi097 ceph-mon[43486]: osd.6 [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] boot 2023-12-14T18:22:36.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:35 smithi097 ceph-mon[43486]: osdmap e43: 7 total, 7 up, 7 in 2023-12-14T18:22:36.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:22:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:36 smithi160 ceph-mon[43840]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:36 smithi160 ceph-mon[43840]: from='client.? 172.21.15.97:0/1168442868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "527a2a02-7c78-4572-9493-54af215f8c7e"}]: dispatch 2023-12-14T18:22:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:36 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "527a2a02-7c78-4572-9493-54af215f8c7e"}]: dispatch 2023-12-14T18:22:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:36 smithi160 ceph-mon[43840]: osdmap e44: 7 total, 7 up, 7 in 2023-12-14T18:22:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:36 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "527a2a02-7c78-4572-9493-54af215f8c7e"}]': finished 2023-12-14T18:22:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:36 smithi160 ceph-mon[43840]: osdmap e45: 8 total, 7 up, 8 in 2023-12-14T18:22:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:36 smithi160 ceph-mon[43840]: from='client.? 172.21.15.97:0/2057556226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:37.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:36 smithi093 ceph-mon[42070]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:37.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:36 smithi093 ceph-mon[42070]: from='client.? 172.21.15.97:0/1168442868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "527a2a02-7c78-4572-9493-54af215f8c7e"}]: dispatch 2023-12-14T18:22:37.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:36 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "527a2a02-7c78-4572-9493-54af215f8c7e"}]: dispatch 2023-12-14T18:22:37.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:36 smithi093 ceph-mon[42070]: osdmap e44: 7 total, 7 up, 7 in 2023-12-14T18:22:37.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:36 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "527a2a02-7c78-4572-9493-54af215f8c7e"}]': finished 2023-12-14T18:22:37.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:36 smithi093 ceph-mon[42070]: osdmap e45: 8 total, 7 up, 8 in 2023-12-14T18:22:37.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:37.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:36 smithi093 ceph-mon[42070]: from='client.? 172.21.15.97:0/2057556226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:36 smithi097 ceph-mon[43486]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 34 MiB used, 536 GiB / 536 GiB avail 2023-12-14T18:22:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:36 smithi097 ceph-mon[43486]: from='client.? 172.21.15.97:0/1168442868' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "527a2a02-7c78-4572-9493-54af215f8c7e"}]: dispatch 2023-12-14T18:22:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:36 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "527a2a02-7c78-4572-9493-54af215f8c7e"}]: dispatch 2023-12-14T18:22:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:36 smithi097 ceph-mon[43486]: osdmap e44: 7 total, 7 up, 7 in 2023-12-14T18:22:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:36 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "527a2a02-7c78-4572-9493-54af215f8c7e"}]': finished 2023-12-14T18:22:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:36 smithi097 ceph-mon[43486]: osdmap e45: 8 total, 7 up, 8 in 2023-12-14T18:22:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:36 smithi097 ceph-mon[43486]: from='client.? 172.21.15.97:0/2057556226' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:39.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:38 smithi093 ceph-mon[42070]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:39.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:38 smithi097 ceph-mon[43486]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:39.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:38 smithi160 ceph-mon[43840]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 39 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:41.288 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:40 smithi097 ceph-mon[43486]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:40 smithi093 ceph-mon[42070]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:40 smithi160 ceph-mon[43840]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:43.264 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:43 smithi097 ceph-mon[43486]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:43.265 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:43 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-14T18:22:43.265 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:43 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:43.265 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:43 smithi097 ceph-mon[43486]: Deploying daemon osd.7 on smithi097 2023-12-14T18:22:43.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:43 smithi093 ceph-mon[42070]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:43.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:43 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-14T18:22:43.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:43 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:43.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:43 smithi093 ceph-mon[42070]: Deploying daemon osd.7 on smithi097 2023-12-14T18:22:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:43 smithi160 ceph-mon[43840]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:43 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2023-12-14T18:22:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:43 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:43 smithi160 ceph-mon[43840]: Deploying daemon osd.7 on smithi097 2023-12-14T18:22:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:45 smithi093 ceph-mon[42070]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:45 smithi097 ceph-mon[43486]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:45.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:45 smithi160 ceph-mon[43840]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:46.939 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:46.939 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:46.939 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:46.939 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:46 smithi097 ceph-mon[43486]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:47.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:47.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:46 smithi093 ceph-mon[42070]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:46 smithi160 ceph-mon[43840]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:48.075 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 7 on host 'smithi097' 2023-12-14T18:22:48.075 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.074+0000 7f31f37fe700 1 -- 172.21.15.97:0/991584015 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3200061a70 con 0x7f31ec0a11d0 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f31ec0a11d0 msgr2=0x7f31ec0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f31ec0a11d0 0x7f31ec0a3680 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3200101380 tx=0x7f31f400a380 comp rx=0 tx=0).stop 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 msgr2=0x7f3200073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 0x7f3200073420 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f31f80062a0 tx=0x7f31f8004d00 comp rx=0 tx=0).stop 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 shutdown_connections 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f31ec0a11d0 0x7f31ec0a3680 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3200100320 0x7f3200074890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3200101eb0 0x7f3200073420 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 --2- 172.21.15.97:0/991584015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3200101520 0x7f3200072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 >> 172.21.15.97:0/991584015 conn(0x7f32000fb970 msgr2=0x7f32001049b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:48.077 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 shutdown_connections 2023-12-14T18:22:48.078 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:22:48.076+0000 7f32086dd700 1 -- 172.21.15.97:0/991584015 wait complete. 2023-12-14T18:22:48.476 DEBUG:teuthology.orchestra.run.smithi097:osd.7> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.7.service 2023-12-14T18:22:48.479 INFO:tasks.cephadm:Deploying osd.8 on smithi160 with /dev/vg_nvme/lv_2... 2023-12-14T18:22:48.479 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2023-12-14T18:22:48.512 INFO:journalctl@ceph.osd.7.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:22:48.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:48 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:48.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:48 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:48.797 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:22:48.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:48.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:48 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:22:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:49 smithi160 ceph-mon[43840]: from='osd.7 [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:22:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:49 smithi160 ceph-mon[43840]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:22:50.088 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_2 2023-12-14T18:22:50.088 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2023-12-14T18:22:50.088 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2023-12-14T18:22:50.088 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2023-12-14T18:22:50.088 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0126549 s, 829 MB/s 2023-12-14T18:22:50.088 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2023-12-14T18:22:50.092 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:22:49 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:22:49.780+0000 7fb186cb2640 -1 osd.7 0 log_to_monitors true 2023-12-14T18:22:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:49 smithi097 ceph-mon[43486]: from='osd.7 [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:22:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:49 smithi097 ceph-mon[43486]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:22:50.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:49 smithi093 ceph-mon[42070]: from='osd.7 [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:22:50.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:49 smithi093 ceph-mon[42070]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2023-12-14T18:22:50.457 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi160:vg_nvme/lv_2 2023-12-14T18:22:50.768 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:22:51.244 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-14T18:22:51.244 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: osdmap e46: 8 total, 7 up, 8 in 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='osd.7 [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi097 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi097 to 5704M 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:51.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:22:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:50 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:22:51.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: osdmap e46: 8 total, 7 up, 8 in 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='osd.7 [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi097 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi097 to 5704M 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:51.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:22:51.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:50 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:22:51.342 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:22:50.852+0000 7fb17c329700 -1 osd.7 0 waiting for initial osdmap 2023-12-14T18:22:51.342 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:22:50.861+0000 7fb17611a700 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: osdmap e46: 8 total, 7 up, 8 in 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='osd.7 [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi097 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi097 to 5704M 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:51.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:22:51.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:51.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:22:51.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:22:51.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:50 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.754+0000 7f6737495700 1 -- 172.21.15.160:0/4117901406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67300ff1e0 msgr2=0x7f67300ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.754+0000 7f6737495700 1 --2- 172.21.15.160:0/4117901406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67300ff1e0 0x7f67300ff5f0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f6724009990 tx=0x7f6724009ca0 comp rx=0 tx=0).stop 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.754+0000 7f6737495700 1 -- 172.21.15.160:0/4117901406 shutdown_connections 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.754+0000 7f6737495700 1 --2- 172.21.15.160:0/4117901406 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6730100d70 0x7f6730107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.754+0000 7f6737495700 1 --2- 172.21.15.160:0/4117901406 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67300ff1e0 0x7f67300ff5f0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.754+0000 7f6737495700 1 --2- 172.21.15.160:0/4117901406 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67301003e0 0x7f6730100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.754+0000 7f6737495700 1 -- 172.21.15.160:0/4117901406 >> 172.21.15.160:0/4117901406 conn(0x7f67300fa7f0 msgr2=0x7f67300fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 -- 172.21.15.160:0/4117901406 shutdown_connections 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 -- 172.21.15.160:0/4117901406 wait complete. 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 Processor -- start 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 -- start start 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300ff1e0 0x7f67301960b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:51.759 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67301003e0 0x7f67301965f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6730100d70 0x7f673019d670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673019dc40 con 0x7f6730100d70 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673019dd80 con 0x7f67300ff1e0 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6737495700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f673019df10 con 0x7f67301003e0 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6734a30700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67301003e0 0x7f67301965f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6734a30700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67301003e0 0x7f67301965f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:48434/0 (socket says 172.21.15.160:48434) 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.755+0000 7f6734a30700 1 -- 172.21.15.160:0/722309967 learned_addr learned my addr 172.21.15.160:0/722309967 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6735a32700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6730100d70 0x7f673019d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6735231700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300ff1e0 0x7f67301960b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6734a30700 1 -- 172.21.15.160:0/722309967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300ff1e0 msgr2=0x7f67301960b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6734a30700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300ff1e0 0x7f67301960b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6734a30700 1 -- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6730100d70 msgr2=0x7f673019d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6734a30700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6730100d70 0x7f673019d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6734a30700 1 -- 172.21.15.160:0/722309967 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67240095f0 con 0x7f67301003e0 2023-12-14T18:22:51.760 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6735231700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300ff1e0 0x7f67301960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:22:51.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6735a32700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6730100d70 0x7f673019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:22:51.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6734a30700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67301003e0 0x7f67301965f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c950 tx=0x7f672c00cd10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:51.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f67227fc700 1 -- 172.21.15.160:0/722309967 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c007ac0 con 0x7f67301003e0 2023-12-14T18:22:51.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f67227fc700 1 -- 172.21.15.160:0/722309967 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f672c004cf0 con 0x7f67301003e0 2023-12-14T18:22:51.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f673010ab30 con 0x7f67301003e0 2023-12-14T18:22:51.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f67227fc700 1 -- 172.21.15.160:0/722309967 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f672c005550 con 0x7f67301003e0 2023-12-14T18:22:51.761 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.756+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f673010a9a0 con 0x7f67301003e0 2023-12-14T18:22:51.762 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.757+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f673019e650 con 0x7f67301003e0 2023-12-14T18:22:51.762 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.758+0000 7f67227fc700 1 -- 172.21.15.160:0/722309967 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f672c004750 con 0x7f67301003e0 2023-12-14T18:22:51.762 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.759+0000 7f67227fc700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f671c0a1150 0x7f671c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:22:51.762 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.759+0000 7f67227fc700 1 -- 172.21.15.160:0/722309967 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 5117+0+0 (secure 0 0 0) 0x7f672c0cbc80 con 0x7f67301003e0 2023-12-14T18:22:51.762 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.759+0000 7f6735231700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f671c0a1150 0x7f671c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:22:51.762 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.760+0000 7f6735231700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f671c0a1150 0x7f671c0a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f672400b5c0 tx=0x7f6724005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:22:51.765 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.763+0000 7f67227fc700 1 -- 172.21.15.160:0/722309967 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f672c096050 con 0x7f67301003e0 2023-12-14T18:22:51.967 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:22:51.965+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7f673019ec00 con 0x7f671c0a1150 2023-12-14T18:22:52.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:51 smithi160 ceph-mon[43840]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:22:52.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:51 smithi160 ceph-mon[43840]: osdmap e47: 8 total, 7 up, 8 in 2023-12-14T18:22:52.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:51 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:52.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:51 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:52.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:51 smithi093 ceph-mon[42070]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:22:52.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:51 smithi093 ceph-mon[42070]: osdmap e47: 8 total, 7 up, 8 in 2023-12-14T18:22:52.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:51 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:52.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:51 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:52.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:51 smithi097 ceph-mon[43486]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:22:52.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:51 smithi097 ceph-mon[43486]: osdmap e47: 8 total, 7 up, 8 in 2023-12-14T18:22:52.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:51 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:52.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:51 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: osd.7 [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] boot 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: osdmap e48: 8 total, 8 up, 8 in 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:53.161 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:53.162 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:52 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:53.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:22:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:22:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: osd.7 [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] boot 2023-12-14T18:22:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: osdmap e48: 8 total, 8 up, 8 in 2023-12-14T18:22:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:53.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:53.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:53.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:53.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:52 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:53.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:22:53.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:22:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: osd.7 [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] boot 2023-12-14T18:22:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: osdmap e48: 8 total, 8 up, 8 in 2023-12-14T18:22:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:22:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 40 MiB used, 626 GiB / 626 GiB avail 2023-12-14T18:22:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: from='client.24274 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:22:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:22:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:22:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:52 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:22:54.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:53 smithi160 ceph-mon[43840]: osdmap e49: 8 total, 8 up, 8 in 2023-12-14T18:22:54.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:53 smithi093 ceph-mon[42070]: osdmap e49: 8 total, 8 up, 8 in 2023-12-14T18:22:54.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:53 smithi097 ceph-mon[43486]: osdmap e49: 8 total, 8 up, 8 in 2023-12-14T18:22:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:54 smithi160 ceph-mon[43840]: osdmap e50: 8 total, 8 up, 8 in 2023-12-14T18:22:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:54 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/200332111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d79000b-a942-43b7-85f7-620e6b98bf90"}]: dispatch 2023-12-14T18:22:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:54 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d79000b-a942-43b7-85f7-620e6b98bf90"}]: dispatch 2023-12-14T18:22:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:54 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d79000b-a942-43b7-85f7-620e6b98bf90"}]': finished 2023-12-14T18:22:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:54 smithi160 ceph-mon[43840]: osdmap e51: 9 total, 8 up, 9 in 2023-12-14T18:22:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:54 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:22:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:54 smithi160 ceph-mon[43840]: pgmap v140: 1 pgs: 1 remapped; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:22:55.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:54 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/908177' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:54 smithi093 ceph-mon[42070]: osdmap e50: 8 total, 8 up, 8 in 2023-12-14T18:22:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:54 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/200332111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d79000b-a942-43b7-85f7-620e6b98bf90"}]: dispatch 2023-12-14T18:22:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:54 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d79000b-a942-43b7-85f7-620e6b98bf90"}]: dispatch 2023-12-14T18:22:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:54 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d79000b-a942-43b7-85f7-620e6b98bf90"}]': finished 2023-12-14T18:22:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:54 smithi093 ceph-mon[42070]: osdmap e51: 9 total, 8 up, 9 in 2023-12-14T18:22:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:54 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:22:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:54 smithi093 ceph-mon[42070]: pgmap v140: 1 pgs: 1 remapped; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:22:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:54 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/908177' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:55.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:54 smithi097 ceph-mon[43486]: osdmap e50: 8 total, 8 up, 8 in 2023-12-14T18:22:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:54 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/200332111' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d79000b-a942-43b7-85f7-620e6b98bf90"}]: dispatch 2023-12-14T18:22:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:54 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5d79000b-a942-43b7-85f7-620e6b98bf90"}]: dispatch 2023-12-14T18:22:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:54 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5d79000b-a942-43b7-85f7-620e6b98bf90"}]': finished 2023-12-14T18:22:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:54 smithi097 ceph-mon[43486]: osdmap e51: 9 total, 8 up, 9 in 2023-12-14T18:22:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:54 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:22:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:54 smithi097 ceph-mon[43486]: pgmap v140: 1 pgs: 1 remapped; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:22:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:54 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/908177' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:22:56.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:55 smithi160 ceph-mon[43840]: osdmap e52: 9 total, 8 up, 9 in 2023-12-14T18:22:56.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:55 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:22:56.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:55 smithi093 ceph-mon[42070]: osdmap e52: 9 total, 8 up, 9 in 2023-12-14T18:22:56.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:55 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:22:56.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:55 smithi097 ceph-mon[43486]: osdmap e52: 9 total, 8 up, 9 in 2023-12-14T18:22:56.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:55 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:22:57.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:56 smithi160 ceph-mon[43840]: pgmap v142: 1 pgs: 1 remapped; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:22:57.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:56 smithi093 ceph-mon[42070]: pgmap v142: 1 pgs: 1 remapped; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:22:57.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:56 smithi097 ceph-mon[43486]: pgmap v142: 1 pgs: 1 remapped; 577 KiB data, 45 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:22:59.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:58 smithi160 ceph-mon[43840]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:22:59.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:58 smithi093 ceph-mon[42070]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:22:59.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:58 smithi097 ceph-mon[43486]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:00.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-14T18:23:00.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:22:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:00.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-14T18:23:00.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:22:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:00.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2023-12-14T18:23:00.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:22:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:01.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:00 smithi160 ceph-mon[43840]: Deploying daemon osd.8 on smithi160 2023-12-14T18:23:01.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:00 smithi160 ceph-mon[43840]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:01.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:00 smithi093 ceph-mon[42070]: Deploying daemon osd.8 on smithi160 2023-12-14T18:23:01.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:00 smithi093 ceph-mon[42070]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:01.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:00 smithi097 ceph-mon[43486]: Deploying daemon osd.8 on smithi160 2023-12-14T18:23:01.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:00 smithi097 ceph-mon[43486]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:03.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:02 smithi160 ceph-mon[43840]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:03.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:02 smithi093 ceph-mon[42070]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:03.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:02 smithi097 ceph-mon[43486]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:04.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:04.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:04.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:04.693 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:04.693 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:04.693 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:04.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:04.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:04.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:05.470 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:05 smithi160 ceph-mon[43840]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:05.470 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:05 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:05.470 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:05 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:05.470 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:05 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:05.471 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:05 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:05.471 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:05 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:05.569 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.567+0000 7f67227fc700 1 -- 172.21.15.160:0/722309967 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f673019ec00 con 0x7f671c0a1150 2023-12-14T18:23:05.569 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 8 on host 'smithi160' 2023-12-14T18:23:05.571 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f671c0a1150 msgr2=0x7f671c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:05.571 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f671c0a1150 0x7f671c0a3600 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f672400b5c0 tx=0x7f6724005fb0 comp rx=0 tx=0).stop 2023-12-14T18:23:05.571 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67301003e0 msgr2=0x7f67301965f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:05.571 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67301003e0 0x7f67301965f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f672c00c950 tx=0x7f672c00cd10 comp rx=0 tx=0).stop 2023-12-14T18:23:05.571 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 shutdown_connections 2023-12-14T18:23:05.571 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f671c0a1150 0x7f671c0a3600 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:05.571 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6730100d70 0x7f673019d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:05.572 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f67300ff1e0 0x7f67301960b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:05.572 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 --2- 172.21.15.160:0/722309967 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f67301003e0 0x7f67301965f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:05.572 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 >> 172.21.15.160:0/722309967 conn(0x7f67300fa7f0 msgr2=0x7f6730068650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:05.572 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 shutdown_connections 2023-12-14T18:23:05.572 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:05.569+0000 7f6737495700 1 -- 172.21.15.160:0/722309967 wait complete. 2023-12-14T18:23:05.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:05 smithi097 ceph-mon[43486]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:05.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:05 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:05.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:05 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:05.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:05 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:05.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:05 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:05.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:05 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:05.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:05 smithi093 ceph-mon[42070]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:05.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:05 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:05.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:05 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:05.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:05 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:05.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:05 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:05.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:05 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:05.952 DEBUG:teuthology.orchestra.run.smithi160:osd.8> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.8.service 2023-12-14T18:23:05.954 INFO:tasks.cephadm:Deploying osd.9 on smithi093 with /dev/vg_nvme/lv_1... 2023-12-14T18:23:05.954 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-14T18:23:06.001 INFO:journalctl@ceph.osd.8.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:23:06.289 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:23:06.425 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:06 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:06.425 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:06 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:23:06.425 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:06 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:06.425 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:06.425 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:06.425 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:06 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:06.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:06 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:06.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:06 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:23:06.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:06 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:06.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:06 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:06.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:06 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:06.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:06 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:06.706 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:06 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:06.706 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:06 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:23:06.706 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:06 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:06.706 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:06 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:06.706 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:06 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:06.707 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:06 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:07.049 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:23:07 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:23:07.046+0000 7faec9cc4640 -1 osd.8 0 log_to_monitors true 2023-12-14T18:23:07.546 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-14T18:23:07.546 INFO:teuthology.orchestra.run.smithi093.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-14T18:23:07.546 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10+0 records in 2023-12-14T18:23:07.546 INFO:teuthology.orchestra.run.smithi093.stderr:10+0 records out 2023-12-14T18:23:07.546 INFO:teuthology.orchestra.run.smithi093.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111698 s, 939 MB/s 2023-12-14T18:23:07.546 INFO:teuthology.orchestra.run.smithi093.stderr:--> Zapping successful for: 2023-12-14T18:23:07.570 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:07 smithi160 ceph-mon[43840]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:07.570 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:07 smithi160 ceph-mon[43840]: from='osd.8 [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:23:07.570 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:07 smithi160 ceph-mon[43840]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:23:07.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:07 smithi093 ceph-mon[42070]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:07.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:07 smithi093 ceph-mon[42070]: from='osd.8 [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:23:07.585 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:07 smithi093 ceph-mon[42070]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:23:07.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:07 smithi097 ceph-mon[43486]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:07.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:07 smithi097 ceph-mon[43486]: from='osd.8 [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:23:07.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:07 smithi097 ceph-mon[43486]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2023-12-14T18:23:07.927 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi093:vg_nvme/lv_1 2023-12-14T18:23:08.234 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:23:08.996 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:23:08.736+0000 7faebf33b700 -1 osd.8 0 waiting for initial osdmap 2023-12-14T18:23:08.996 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:23:08.749+0000 7faeb7128700 -1 osd.8 54 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:23:08.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi160 2023-12-14T18:23:08.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:08.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:08.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:08.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:08.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:08.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi160 to 5704M 2023-12-14T18:23:08.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: osdmap e53: 9 total, 8 up, 9 in 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='osd.8 [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:23:08.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:08 smithi160 ceph-mon[43840]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:09.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi160 2023-12-14T18:23:09.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:09.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:09.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:09.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:09.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:09.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi160 to 5704M 2023-12-14T18:23:09.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: osdmap e53: 9 total, 8 up, 9 in 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='osd.8 [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:23:09.008 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:08 smithi093 ceph-mon[42070]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:09.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi160 2023-12-14T18:23:09.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:09.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:09.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:09.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:09.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:09.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi160 to 5704M 2023-12-14T18:23:09.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:09.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: osdmap e53: 9 total, 8 up, 9 in 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='osd.8 [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:23:09.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:08 smithi097 ceph-mon[43486]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:09.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.175+0000 7f871a788700 1 -- 172.21.15.93:0/1003226943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8714100d70 msgr2=0x7f8714107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:09.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.175+0000 7f871a788700 1 --2- 172.21.15.93:0/1003226943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8714100d70 0x7f8714107630 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f870c00b0a0 tx=0x7f870c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.175+0000 7f871a788700 1 -- 172.21.15.93:0/1003226943 shutdown_connections 2023-12-14T18:23:09.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.175+0000 7f871a788700 1 --2- 172.21.15.93:0/1003226943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8714100d70 0x7f8714107630 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.175+0000 7f871a788700 1 --2- 172.21.15.93:0/1003226943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87140ff1e0 0x7f87140ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.175+0000 7f871a788700 1 --2- 172.21.15.93:0/1003226943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87141003e0 0x7f8714100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.175+0000 7f871a788700 1 -- 172.21.15.93:0/1003226943 >> 172.21.15.93:0/1003226943 conn(0x7f87140fa7f0 msgr2=0x7f87140fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:09.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.175+0000 7f871a788700 1 -- 172.21.15.93:0/1003226943 shutdown_connections 2023-12-14T18:23:09.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.175+0000 7f871a788700 1 -- 172.21.15.93:0/1003226943 wait complete. 2023-12-14T18:23:09.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f871a788700 1 Processor -- start 2023-12-14T18:23:09.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f871a788700 1 -- start start 2023-12-14T18:23:09.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f871a788700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87140ff1e0 0x7f8714196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:09.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f871a788700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87141003e0 0x7f8714196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:09.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f871a788700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8714100d70 0x7f871419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:09.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f871a788700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871419dbb0 con 0x7f87140ff1e0 2023-12-14T18:23:09.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f871a788700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871419dcf0 con 0x7f87141003e0 2023-12-14T18:23:09.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f871a788700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f871419de80 con 0x7f8714100d70 2023-12-14T18:23:09.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f8713fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87140ff1e0 0x7f8714196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f8713fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87140ff1e0 0x7f8714196020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46384/0 (socket says 172.21.15.93:46384) 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f8713fff700 1 -- 172.21.15.93:0/4271436450 learned_addr learned my addr 172.21.15.93:0/4271436450 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f87137fe700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87141003e0 0x7f8714196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f8718d25700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8714100d70 0x7f871419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.176+0000 7f8713fff700 1 -- 172.21.15.93:0/4271436450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8714100d70 msgr2=0x7f871419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f8713fff700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8714100d70 0x7f871419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f8713fff700 1 -- 172.21.15.93:0/4271436450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87141003e0 msgr2=0x7f8714196560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f8713fff700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87141003e0 0x7f8714196560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f8713fff700 1 -- 172.21.15.93:0/4271436450 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f870c009d00 con 0x7f87140ff1e0 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f8718d25700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8714100d70 0x7f871419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:23:09.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f8713fff700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87140ff1e0 0x7f8714196020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f870400e9a0 tx=0x7f870400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:09.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f87117fa700 1 -- 172.21.15.93:0/4271436450 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704009790 con 0x7f87140ff1e0 2023-12-14T18:23:09.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f87117fa700 1 -- 172.21.15.93:0/4271436450 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87040044e0 con 0x7f87140ff1e0 2023-12-14T18:23:09.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f871410ab30 con 0x7f87140ff1e0 2023-12-14T18:23:09.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f87117fa700 1 -- 172.21.15.93:0/4271436450 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8704003d20 con 0x7f87140ff1e0 2023-12-14T18:23:09.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.177+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f871410a9a0 con 0x7f87140ff1e0 2023-12-14T18:23:09.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.178+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8714066ea0 con 0x7f87140ff1e0 2023-12-14T18:23:09.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.179+0000 7f87117fa700 1 -- 172.21.15.93:0/4271436450 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8704003e80 con 0x7f87140ff1e0 2023-12-14T18:23:09.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.179+0000 7f87117fa700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87000a1150 0x7f87000a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:09.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.180+0000 7f87137fe700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87000a1150 0x7f87000a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:09.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.180+0000 7f87117fa700 1 -- 172.21.15.93:0/4271436450 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(54..54 src has 1..54) v4 ==== 5549+0+0 (secure 0 0 0) 0x7f8704014070 con 0x7f87140ff1e0 2023-12-14T18:23:09.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.180+0000 7f87137fe700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87000a1150 0x7f87000a3600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00ac50 tx=0x7f86fc00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:09.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.184+0000 7f87117fa700 1 -- 172.21.15.93:0/4271436450 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8704096300 con 0x7f87140ff1e0 2023-12-14T18:23:09.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:09.382+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f87140695d0 con 0x7f87000a1150 2023-12-14T18:23:09.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:09 smithi160 ceph-mon[43840]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:23:09.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:09 smithi160 ceph-mon[43840]: osdmap e54: 9 total, 8 up, 9 in 2023-12-14T18:23:09.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:09 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:09.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:09 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:09.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:09 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:09.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:09 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:09.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:09 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:10.006 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:09 smithi093 ceph-mon[42070]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:23:10.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:09 smithi093 ceph-mon[42070]: osdmap e54: 9 total, 8 up, 9 in 2023-12-14T18:23:10.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:09 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:10.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:09 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:10.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:09 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:10.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:09 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:10.007 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:09 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:10.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:09 smithi097 ceph-mon[43486]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:23:10.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:09 smithi097 ceph-mon[43486]: osdmap e54: 9 total, 8 up, 9 in 2023-12-14T18:23:10.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:09 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:10.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:09 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:10.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:09 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:10.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:09 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:10.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:09 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:10.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:10 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:23:10.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:10 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:23:10.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:10 smithi160 ceph-mon[43840]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:10.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:10.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:10 smithi160 ceph-mon[43840]: osd.8 [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] boot 2023-12-14T18:23:10.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:10 smithi160 ceph-mon[43840]: osdmap e55: 9 total, 9 up, 9 in 2023-12-14T18:23:10.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:10.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:10 smithi160 ceph-mon[43840]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:10.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:10 smithi160 ceph-mon[43840]: osdmap e56: 9 total, 9 up, 9 in 2023-12-14T18:23:11.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:10 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:23:11.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:10 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:23:11.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:10 smithi093 ceph-mon[42070]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:11.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:11.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:10 smithi093 ceph-mon[42070]: osd.8 [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] boot 2023-12-14T18:23:11.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:10 smithi093 ceph-mon[42070]: osdmap e55: 9 total, 9 up, 9 in 2023-12-14T18:23:11.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:11.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:10 smithi093 ceph-mon[42070]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:11.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:10 smithi093 ceph-mon[42070]: osdmap e56: 9 total, 9 up, 9 in 2023-12-14T18:23:11.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:10 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:23:11.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:10 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:23:11.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:10 smithi097 ceph-mon[43486]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi093:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:11.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:11.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:10 smithi097 ceph-mon[43486]: osd.8 [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] boot 2023-12-14T18:23:11.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:10 smithi097 ceph-mon[43486]: osdmap e55: 9 total, 9 up, 9 in 2023-12-14T18:23:11.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:23:11.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:10 smithi097 ceph-mon[43486]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 46 MiB used, 715 GiB / 715 GiB avail 2023-12-14T18:23:11.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:10 smithi097 ceph-mon[43486]: osdmap e56: 9 total, 9 up, 9 in 2023-12-14T18:23:11.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:11 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1945694213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14e038ed-c6e1-4d19-88cc-329ffe2343e9"}]: dispatch 2023-12-14T18:23:11.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:11 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1945694213' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14e038ed-c6e1-4d19-88cc-329ffe2343e9"}]': finished 2023-12-14T18:23:11.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:11 smithi160 ceph-mon[43840]: osdmap e57: 10 total, 9 up, 10 in 2023-12-14T18:23:11.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:11 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:12.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:11 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1945694213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14e038ed-c6e1-4d19-88cc-329ffe2343e9"}]: dispatch 2023-12-14T18:23:12.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:11 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1945694213' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14e038ed-c6e1-4d19-88cc-329ffe2343e9"}]': finished 2023-12-14T18:23:12.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:11 smithi093 ceph-mon[42070]: osdmap e57: 10 total, 9 up, 10 in 2023-12-14T18:23:12.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:11 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:12.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:11 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1945694213' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "14e038ed-c6e1-4d19-88cc-329ffe2343e9"}]: dispatch 2023-12-14T18:23:12.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:11 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1945694213' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "14e038ed-c6e1-4d19-88cc-329ffe2343e9"}]': finished 2023-12-14T18:23:12.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:11 smithi097 ceph-mon[43486]: osdmap e57: 10 total, 9 up, 10 in 2023-12-14T18:23:12.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:11 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:13.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:12 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2293523294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:13.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:12 smithi093 ceph-mon[42070]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:13.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:12 smithi093 ceph-mon[42070]: osdmap e58: 10 total, 9 up, 10 in 2023-12-14T18:23:13.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:12 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:13.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:12 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2293523294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:13.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:12 smithi097 ceph-mon[43486]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:13.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:12 smithi097 ceph-mon[43486]: osdmap e58: 10 total, 9 up, 10 in 2023-12-14T18:23:13.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:12 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:13.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:12 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2293523294' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:13.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:12 smithi160 ceph-mon[43840]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:13.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:12 smithi160 ceph-mon[43840]: osdmap e58: 10 total, 9 up, 10 in 2023-12-14T18:23:13.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:12 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:14.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:14 smithi160 ceph-mon[43840]: osdmap e59: 10 total, 9 up, 10 in 2023-12-14T18:23:14.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:14 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:14.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:14 smithi093 ceph-mon[42070]: osdmap e59: 10 total, 9 up, 10 in 2023-12-14T18:23:14.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:14 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:14.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:14 smithi097 ceph-mon[43486]: osdmap e59: 10 total, 9 up, 10 in 2023-12-14T18:23:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:14 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:15.495 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:15 smithi160 ceph-mon[43840]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:23:15.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:15 smithi093 ceph-mon[42070]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:23:15.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:15 smithi097 ceph-mon[43486]: pgmap v158: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:23:16.988 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:16 smithi093 ceph-mon[42070]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-14T18:23:16.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:16 smithi160 ceph-mon[43840]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-14T18:23:17.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:16 smithi097 ceph-mon[43486]: pgmap v159: 1 pgs: 1 peering; 577 KiB data, 52 MiB used, 805 GiB / 805 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-14T18:23:17.947 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-14T18:23:17.947 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:17.947 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:17 smithi093 ceph-mon[42070]: Deploying daemon osd.9 on smithi093 2023-12-14T18:23:17.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-14T18:23:17.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:17.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:17 smithi160 ceph-mon[43840]: Deploying daemon osd.9 on smithi093 2023-12-14T18:23:18.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2023-12-14T18:23:18.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:18.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:17 smithi097 ceph-mon[43486]: Deploying daemon osd.9 on smithi093 2023-12-14T18:23:18.966 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:18 smithi093 ceph-mon[42070]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-14T18:23:18.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:18 smithi160 ceph-mon[43840]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-14T18:23:19.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:18 smithi097 ceph-mon[43486]: pgmap v160: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 86 KiB/s, 0 objects/s recovering 2023-12-14T18:23:21.237 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:20 smithi093 ceph-mon[42070]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-14T18:23:21.237 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:21.237 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:21.238 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:21.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:20 smithi160 ceph-mon[43840]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-14T18:23:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:20 smithi097 ceph-mon[43486]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2023-12-14T18:23:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:23.165 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:22 smithi093 ceph-mon[42070]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-12-14T18:23:23.166 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:23.166 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:23.166 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:23.166 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:23.166 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:22 smithi160 ceph-mon[43840]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-12-14T18:23:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:23.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.281+0000 7f87117fa700 1 -- 172.21.15.93:0/4271436450 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f87140695d0 con 0x7f87000a1150 2023-12-14T18:23:23.283 INFO:teuthology.orchestra.run.smithi093.stdout:Created osd(s) 9 on host 'smithi093' 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87000a1150 msgr2=0x7f87000a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87000a1150 0x7f87000a3600 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f86fc00ac50 tx=0x7f86fc00a380 comp rx=0 tx=0).stop 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87140ff1e0 msgr2=0x7f8714196020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87140ff1e0 0x7f8714196020 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f870400e9a0 tx=0x7f870400ecb0 comp rx=0 tx=0).stop 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 shutdown_connections 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87000a1150 0x7f87000a3600 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87140ff1e0 0x7f8714196020 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8714100d70 0x7f871419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 --2- 172.21.15.93:0/4271436450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87141003e0 0x7f8714196560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:23.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 >> 172.21.15.93:0/4271436450 conn(0x7f87140fa7f0 msgr2=0x7f8714068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:23.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 shutdown_connections 2023-12-14T18:23:23.286 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:23:23.283+0000 7f871a788700 1 -- 172.21.15.93:0/4271436450 wait complete. 2023-12-14T18:23:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:22 smithi097 ceph-mon[43486]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail; 59 KiB/s, 0 objects/s recovering 2023-12-14T18:23:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:23.654 DEBUG:teuthology.orchestra.run.smithi093:osd.9> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.9.service 2023-12-14T18:23:23.656 INFO:tasks.cephadm:Deploying osd.10 on smithi097 with /dev/vg_nvme/lv_1... 2023-12-14T18:23:23.657 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-14T18:23:23.688 INFO:journalctl@ceph.osd.9.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:23:23.986 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:23:24.408 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:24 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:24.409 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:24 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:23:24.409 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:24 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:24.409 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:24 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:24.409 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:24 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:24.409 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:24 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:24.429 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:24 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:24.429 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:24 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:23:24.429 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:24 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:24.430 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:24 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:24.430 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:24 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:24.430 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:24 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:24 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:24 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:23:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:24 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:24 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:24 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:24.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:24 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:25.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:23:24 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:23:24.749+0000 7fd7e1ece640 -1 osd.9 0 log_to_monitors true 2023-12-14T18:23:25.186 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-14T18:23:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-14T18:23:25.187 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 10+0 records in 2023-12-14T18:23:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:10+0 records out 2023-12-14T18:23:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113055 s, 927 MB/s 2023-12-14T18:23:25.187 INFO:teuthology.orchestra.run.smithi097.stderr: stderr: 2023-12-14T18:23:25.187 INFO:teuthology.orchestra.run.smithi097.stderr:--> Zapping successful for: 2023-12-14T18:23:25.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:25 smithi097 ceph-mon[43486]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:25.486 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:25 smithi097 ceph-mon[43486]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-14T18:23:25.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:25 smithi093 ceph-mon[42070]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:25 smithi093 ceph-mon[42070]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-14T18:23:25.579 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi097:vg_nvme/lv_1 2023-12-14T18:23:25.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:25 smithi160 ceph-mon[43840]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:25.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:25 smithi160 ceph-mon[43840]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2023-12-14T18:23:25.884 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:23:26.341 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-14T18:23:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: osdmap e60: 10 total, 9 up, 10 in 2023-12-14T18:23:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:23:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi093 2023-12-14T18:23:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi093 to 4278M 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:23:26.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:26 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:26.565 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:23:26.259+0000 7fd7d7545700 -1 osd.9 0 waiting for initial osdmap 2023-12-14T18:23:26.565 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:23:26.272+0000 7fd7d1b37700 -1 osd.9 61 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:23:26.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-14T18:23:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: osdmap e60: 10 total, 9 up, 10 in 2023-12-14T18:23:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:23:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi093 2023-12-14T18:23:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi093 to 4278M 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:23:26.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:26 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:26.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2023-12-14T18:23:26.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: osdmap e60: 10 total, 9 up, 10 in 2023-12-14T18:23:26.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]: dispatch 2023-12-14T18:23:26.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:26.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi093 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi093 to 4278M 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:26.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:26.748 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:26.748 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:23:26.748 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:26 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:26.847 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 -- 172.21.15.97:0/897814660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 msgr2=0x7fdb9c0ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/897814660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 0x7fdb9c0ff830 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009990 tx=0x7fdb98009ca0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 -- 172.21.15.97:0/897814660 shutdown_connections 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/897814660 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb9c100fb0 0x7fdb9c107870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/897814660 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 0x7fdb9c0ff830 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/897814660 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb9c100620 0x7fdb9c100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 -- 172.21.15.97:0/897814660 >> 172.21.15.97:0/897814660 conn(0x7fdb9c0faa50 msgr2=0x7fdb9c0fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 -- 172.21.15.97:0/897814660 shutdown_connections 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 -- 172.21.15.97:0/897814660 wait complete. 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 Processor -- start 2023-12-14T18:23:26.848 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 -- start start 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 0x7fdb9c1963b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb9c100620 0x7fdb9c1968f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb9c100fb0 0x7fdb9c19d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c19df40 con 0x7fdb9c100fb0 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c19e080 con 0x7fdb9c0ff420 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.843+0000 7fdba3a2c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9c19e210 con 0x7fdb9c100620 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba17c8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 0x7fdb9c1963b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba17c8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 0x7fdb9c1963b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.97:60240/0 (socket says 172.21.15.97:60240) 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba17c8700 1 -- 172.21.15.97:0/325315764 learned_addr learned my addr 172.21.15.97:0/325315764 (peer_addr_for_me v2:172.21.15.97:0/0) 2023-12-14T18:23:26.849 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba0fc7700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb9c100620 0x7fdb9c1968f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba17c8700 1 -- 172.21.15.97:0/325315764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb9c100620 msgr2=0x7fdb9c1968f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba17c8700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb9c100620 0x7fdb9c1968f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba17c8700 1 -- 172.21.15.97:0/325315764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb9c100fb0 msgr2=0x7fdb9c19d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba17c8700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb9c100fb0 0x7fdb9c19d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba17c8700 1 -- 172.21.15.97:0/325315764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb980095f0 con 0x7fdb9c0ff420 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba0fc7700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb9c100620 0x7fdb9c1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba17c8700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 0x7fdb9c1963b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb98000c00 tx=0x7fdb9800f7b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdb927fc700 1 -- 172.21.15.97:0/325315764 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9801d070 con 0x7fdb9c0ff420 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdb927fc700 1 -- 172.21.15.97:0/325315764 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb9800fca0 con 0x7fdb9c0ff420 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb9c10ae70 con 0x7fdb9c0ff420 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdb927fc700 1 -- 172.21.15.97:0/325315764 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb98017720 con 0x7fdb9c0ff420 2023-12-14T18:23:26.850 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.844+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb9c10ace0 con 0x7fdb9c0ff420 2023-12-14T18:23:26.851 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.845+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb9c067b00 con 0x7fdb9c0ff420 2023-12-14T18:23:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.846+0000 7fdb927fc700 1 -- 172.21.15.97:0/325315764 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdb98017880 con 0x7fdb9c0ff420 2023-12-14T18:23:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.847+0000 7fdb927fc700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdb880a1150 0x7fdb880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.847+0000 7fdb927fc700 1 -- 172.21.15.97:0/325315764 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5981+0+0 (secure 0 0 0) 0x7fdb980cd9d0 con 0x7fdb9c0ff420 2023-12-14T18:23:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.847+0000 7fdba0fc7700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdb880a1150 0x7fdb880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.848+0000 7fdba0fc7700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdb880a1150 0x7fdb880a3600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c007900 tx=0x7fdb8c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:26.852 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:26.850+0000 7fdb927fc700 1 -- 172.21.15.97:0/325315764 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdb98097ad0 con 0x7fdb9c0ff420 2023-12-14T18:23:27.048 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:27.046+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fdb9c069790 con 0x7fdb880a1150 2023-12-14T18:23:27.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:27 smithi097 ceph-mon[43486]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:27.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:27 smithi097 ceph-mon[43486]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:23:27.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:27 smithi097 ceph-mon[43486]: osdmap e61: 10 total, 9 up, 10 in 2023-12-14T18:23:27.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:27 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:27.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:27 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:27.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:27 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:27.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:27 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:27.547 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:27 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:27 smithi093 ceph-mon[42070]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:27 smithi093 ceph-mon[42070]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:23:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:27 smithi093 ceph-mon[42070]: osdmap e61: 10 total, 9 up, 10 in 2023-12-14T18:23:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:27 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:27 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:27 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:27 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:27 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:27.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:27 smithi160 ceph-mon[43840]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:27.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:27 smithi160 ceph-mon[43840]: from='osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi093", "root=default"]}]': finished 2023-12-14T18:23:27.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:27 smithi160 ceph-mon[43840]: osdmap e61: 10 total, 9 up, 10 in 2023-12-14T18:23:27.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:27 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:27.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:27 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:27.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:27 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:27.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:27 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:27.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:27 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:28.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:28 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:23:28.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:28 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:23:28.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:28 smithi093 ceph-mon[42070]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:28.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:28 smithi093 ceph-mon[42070]: osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] boot 2023-12-14T18:23:28.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:28 smithi093 ceph-mon[42070]: osdmap e62: 10 total, 10 up, 10 in 2023-12-14T18:23:28.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:28.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:28 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:23:28.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:28 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:23:28.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:28 smithi097 ceph-mon[43486]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:28.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:28 smithi097 ceph-mon[43486]: osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] boot 2023-12-14T18:23:28.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:28 smithi097 ceph-mon[43486]: osdmap e62: 10 total, 10 up, 10 in 2023-12-14T18:23:28.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:28 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:23:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:28 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:23:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:28 smithi160 ceph-mon[43840]: from='client.24323 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi097:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:28 smithi160 ceph-mon[43840]: osd.9 [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] boot 2023-12-14T18:23:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:28 smithi160 ceph-mon[43840]: osdmap e62: 10 total, 10 up, 10 in 2023-12-14T18:23:28.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:23:29.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:29 smithi093 ceph-mon[42070]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:29.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:29 smithi093 ceph-mon[42070]: osdmap e63: 10 total, 10 up, 10 in 2023-12-14T18:23:29.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:29 smithi093 ceph-mon[42070]: from='client.? 172.21.15.97:0/2417915828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44df1de1-facd-4592-855a-d32c456cf8ee"}]: dispatch 2023-12-14T18:23:29.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:29 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44df1de1-facd-4592-855a-d32c456cf8ee"}]: dispatch 2023-12-14T18:23:29.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:29 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44df1de1-facd-4592-855a-d32c456cf8ee"}]': finished 2023-12-14T18:23:29.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:29 smithi093 ceph-mon[42070]: osdmap e64: 11 total, 10 up, 11 in 2023-12-14T18:23:29.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:29 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:29.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:29 smithi097 ceph-mon[43486]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:29.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:29 smithi097 ceph-mon[43486]: osdmap e63: 10 total, 10 up, 10 in 2023-12-14T18:23:29.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:29 smithi097 ceph-mon[43486]: from='client.? 172.21.15.97:0/2417915828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44df1de1-facd-4592-855a-d32c456cf8ee"}]: dispatch 2023-12-14T18:23:29.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:29 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44df1de1-facd-4592-855a-d32c456cf8ee"}]: dispatch 2023-12-14T18:23:29.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:29 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44df1de1-facd-4592-855a-d32c456cf8ee"}]': finished 2023-12-14T18:23:29.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:29 smithi097 ceph-mon[43486]: osdmap e64: 11 total, 10 up, 11 in 2023-12-14T18:23:29.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:29 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:29.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:29 smithi160 ceph-mon[43840]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 53 MiB used, 805 GiB / 805 GiB avail 2023-12-14T18:23:29.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:29 smithi160 ceph-mon[43840]: osdmap e63: 10 total, 10 up, 10 in 2023-12-14T18:23:29.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:29 smithi160 ceph-mon[43840]: from='client.? 172.21.15.97:0/2417915828' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44df1de1-facd-4592-855a-d32c456cf8ee"}]: dispatch 2023-12-14T18:23:29.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:29 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "44df1de1-facd-4592-855a-d32c456cf8ee"}]: dispatch 2023-12-14T18:23:29.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:29 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "44df1de1-facd-4592-855a-d32c456cf8ee"}]': finished 2023-12-14T18:23:29.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:29 smithi160 ceph-mon[43840]: osdmap e64: 11 total, 10 up, 11 in 2023-12-14T18:23:29.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:29 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:30 smithi093 ceph-mon[42070]: from='client.? 172.21.15.97:0/4169004489' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:30.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:30 smithi097 ceph-mon[43486]: from='client.? 172.21.15.97:0/4169004489' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:30 smithi160 ceph-mon[43840]: from='client.? 172.21.15.97:0/4169004489' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:31 smithi093 ceph-mon[42070]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:31.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:31 smithi097 ceph-mon[43486]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:31.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:31 smithi160 ceph-mon[43840]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 58 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:33 smithi093 ceph-mon[42070]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:33.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:33 smithi097 ceph-mon[43486]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:33.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:33 smithi160 ceph-mon[43840]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:35 smithi093 ceph-mon[42070]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-14T18:23:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:35.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:35 smithi097 ceph-mon[43486]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-14T18:23:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:35 smithi160 ceph-mon[43840]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2023-12-14T18:23:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:36.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:36 smithi093 ceph-mon[42070]: Deploying daemon osd.10 on smithi097 2023-12-14T18:23:36.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:36 smithi097 ceph-mon[43486]: Deploying daemon osd.10 on smithi097 2023-12-14T18:23:36.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:36 smithi160 ceph-mon[43840]: Deploying daemon osd.10 on smithi097 2023-12-14T18:23:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:37 smithi093 ceph-mon[42070]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:37.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:37 smithi097 ceph-mon[43486]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:37.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:37 smithi160 ceph-mon[43840]: pgmap v174: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:39.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:39 smithi093 ceph-mon[42070]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:39.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:39 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:39.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:39 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:39.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:39 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:39.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:39 smithi097 ceph-mon[43486]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:39.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:39 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:39.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:39 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:39.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:39 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:39.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:39 smithi160 ceph-mon[43840]: pgmap v175: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:39.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:39 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:39.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:39 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:39.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:39 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:41.054 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.052+0000 7fdb927fc700 1 -- 172.21.15.97:0/325315764 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7fdb9c069790 con 0x7fdb880a1150 2023-12-14T18:23:41.054 INFO:teuthology.orchestra.run.smithi097.stdout:Created osd(s) 10 on host 'smithi097' 2023-12-14T18:23:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.054+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdb880a1150 msgr2=0x7fdb880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.054+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdb880a1150 0x7fdb880a3600 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fdb8c007900 tx=0x7fdb8c008040 comp rx=0 tx=0).stop 2023-12-14T18:23:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 msgr2=0x7fdb9c1963b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 0x7fdb9c1963b0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fdb98000c00 tx=0x7fdb9800f7b0 comp rx=0 tx=0).stop 2023-12-14T18:23:41.056 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 shutdown_connections 2023-12-14T18:23:41.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdb880a1150 0x7fdb880a3600 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:41.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdb9c100fb0 0x7fdb9c19d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:41.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdb9c0ff420 0x7fdb9c1963b0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:41.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 --2- 172.21.15.97:0/325315764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdb9c100620 0x7fdb9c1968f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:41.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 >> 172.21.15.97:0/325315764 conn(0x7fdb9c0faa50 msgr2=0x7fdb9c069080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:41.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 shutdown_connections 2023-12-14T18:23:41.057 INFO:teuthology.orchestra.run.smithi097.stderr:2023-12-14T18:23:41.055+0000 7fdba3a2c700 1 -- 172.21.15.97:0/325315764 wait complete. 2023-12-14T18:23:41.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:41.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.273 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:41.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:41.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:41.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:23:41.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:41.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.274 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:41 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:41 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.464 DEBUG:teuthology.orchestra.run.smithi097:osd.10> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.10.service 2023-12-14T18:23:41.467 INFO:tasks.cephadm:Deploying osd.11 on smithi160 with /dev/vg_nvme/lv_1... 2023-12-14T18:23:41.467 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2023-12-14T18:23:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:23:41.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:41.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:41 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:41.500 INFO:journalctl@ceph.osd.10.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:23:41.785 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:23:42.203 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:42 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:42.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:42 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:42.324 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:42 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:43.036 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:23:42 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:23:42.672+0000 7fee8d549640 -1 osd.10 0 log_to_monitors true 2023-12-14T18:23:43.088 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping: /dev/vg_nvme/lv_1 2023-12-14T18:23:43.088 INFO:teuthology.orchestra.run.smithi160.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2023-12-14T18:23:43.088 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10+0 records in 2023-12-14T18:23:43.088 INFO:teuthology.orchestra.run.smithi160.stderr:10+0 records out 2023-12-14T18:23:43.088 INFO:teuthology.orchestra.run.smithi160.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011174 s, 938 MB/s 2023-12-14T18:23:43.088 INFO:teuthology.orchestra.run.smithi160.stderr:--> Zapping successful for: 2023-12-14T18:23:43.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:43 smithi093 ceph-mon[42070]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:43.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:43 smithi093 ceph-mon[42070]: from='osd.10 [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:23:43.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:43 smithi093 ceph-mon[42070]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:23:43.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:43 smithi097 ceph-mon[43486]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:43.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:43 smithi097 ceph-mon[43486]: from='osd.10 [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:23:43.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:43 smithi097 ceph-mon[43486]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:23:43.394 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:43 smithi160 ceph-mon[43840]: pgmap v177: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:43.394 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:43 smithi160 ceph-mon[43840]: from='osd.10 [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:23:43.394 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:43 smithi160 ceph-mon[43840]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2023-12-14T18:23:43.481 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch daemon add osd smithi160:vg_nvme/lv_1 2023-12-14T18:23:43.829 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:23:44.046 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-14T18:23:44.046 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: osdmap e65: 11 total, 10 up, 11 in 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='osd.10 [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi097 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi097 to 4278M 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:44.047 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.048 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:23:44.048 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:44.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-14T18:23:44.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: osdmap e65: 11 total, 10 up, 11 in 2023-12-14T18:23:44.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:44.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='osd.10 [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:23:44.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi097 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi097 to 4278M 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:23:44.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: osdmap e65: 11 total, 10 up, 11 in 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='osd.10 [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]: dispatch 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi097 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:23:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi097 to 4278M 2023-12-14T18:23:44.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:44.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:44.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:44.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:44.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:23:44.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:44.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:23:44.047+0000 7fee82bc0700 -1 osd.10 0 waiting for initial osdmap 2023-12-14T18:23:44.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:23:44 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:23:44.062+0000 7fee7c1b0700 -1 osd.10 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:23:44.839 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.838+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905148039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 msgr2=0x7f2a440ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:44.840 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.838+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905148039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 0x7f2a440ff5f0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2a34009990 tx=0x7f2a34009ca0 comp rx=0 tx=0).stop 2023-12-14T18:23:44.840 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.838+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905148039 shutdown_connections 2023-12-14T18:23:44.840 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.838+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905148039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a44100d70 0x7f2a44107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:44.840 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.838+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905148039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 0x7f2a440ff5f0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:44.840 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.838+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905148039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a441003e0 0x7f2a44100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:44.840 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.838+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905148039 >> 172.21.15.160:0/2905148039 conn(0x7f2a440fa7f0 msgr2=0x7f2a440fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:44.840 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.838+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905148039 shutdown_connections 2023-12-14T18:23:44.840 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.838+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905148039 wait complete. 2023-12-14T18:23:44.843 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a48a22700 1 Processor -- start 2023-12-14T18:23:44.843 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a48a22700 1 -- start start 2023-12-14T18:23:44.843 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a48a22700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 0x7f2a44196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:44.843 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a48a22700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a441003e0 0x7f2a44196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:44.843 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a48a22700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a44100d70 0x7f2a4419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:44.843 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a48a22700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419dbd0 con 0x7f2a44100d70 2023-12-14T18:23:44.843 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a48a22700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419dd10 con 0x7f2a441003e0 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a48a22700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a4419dea0 con 0x7f2a440ff1e0 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a4259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 0x7f2a44196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a4259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 0x7f2a44196040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:44602/0 (socket says 172.21.15.160:44602) 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a4259c700 1 -- 172.21.15.160:0/2905209324 learned_addr learned my addr 172.21.15.160:0/2905209324 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a41d9b700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a441003e0 0x7f2a44196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a4259c700 1 -- 172.21.15.160:0/2905209324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a441003e0 msgr2=0x7f2a44196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a4259c700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a441003e0 0x7f2a44196580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a4259c700 1 -- 172.21.15.160:0/2905209324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a44100d70 msgr2=0x7f2a4419d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a4259c700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a44100d70 0x7f2a4419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:44.844 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a4259c700 1 -- 172.21.15.160:0/2905209324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a340095f0 con 0x7f2a440ff1e0 2023-12-14T18:23:44.845 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a4259c700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 0x7f2a44196040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b5c0 tx=0x7f2a3400f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:44.845 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.839+0000 7f2a337fe700 1 -- 172.21.15.160:0/2905209324 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a3401d070 con 0x7f2a440ff1e0 2023-12-14T18:23:44.845 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.840+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a4410bc80 con 0x7f2a440ff1e0 2023-12-14T18:23:44.845 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.840+0000 7f2a337fe700 1 -- 172.21.15.160:0/2905209324 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2a34022470 con 0x7f2a440ff1e0 2023-12-14T18:23:44.845 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.840+0000 7f2a337fe700 1 -- 172.21.15.160:0/2905209324 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a34017630 con 0x7f2a440ff1e0 2023-12-14T18:23:44.845 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.840+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a4410baf0 con 0x7f2a440ff1e0 2023-12-14T18:23:44.845 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.841+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a441067f0 con 0x7f2a440ff1e0 2023-12-14T18:23:44.845 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.842+0000 7f2a337fe700 1 -- 172.21.15.160:0/2905209324 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2a340228e0 con 0x7f2a440ff1e0 2023-12-14T18:23:44.846 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.842+0000 7f2a337fe700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2a2c0a1150 0x7f2a2c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:23:44.846 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.842+0000 7f2a337fe700 1 -- 172.21.15.160:0/2905209324 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6414+0+0 (secure 0 0 0) 0x7f2a340cd3e0 con 0x7f2a440ff1e0 2023-12-14T18:23:44.847 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.843+0000 7f2a41d9b700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2a2c0a1150 0x7f2a2c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:23:44.847 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.843+0000 7f2a41d9b700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2a2c0a1150 0x7f2a2c0a3600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2a38005950 tx=0x7f2a380058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:23:44.848 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:44.847+0000 7f2a337fe700 1 -- 172.21.15.160:0/2905209324 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2a3405c730 con 0x7f2a440ff1e0 2023-12-14T18:23:45.044 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:45.043+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f2a44061a10 con 0x7f2a2c0a1150 2023-12-14T18:23:45.055 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:45 smithi160 ceph-mon[43840]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:45.056 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:45 smithi160 ceph-mon[43840]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:23:45.056 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:45 smithi160 ceph-mon[43840]: osdmap e66: 11 total, 10 up, 11 in 2023-12-14T18:23:45.056 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:45 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:45.056 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:45 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:45 smithi093 ceph-mon[42070]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:45 smithi093 ceph-mon[42070]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:23:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:45 smithi093 ceph-mon[42070]: osdmap e66: 11 total, 10 up, 11 in 2023-12-14T18:23:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:45 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:45.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:45 smithi097 ceph-mon[43486]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:45 smithi097 ceph-mon[43486]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi097", "root=default"]}]': finished 2023-12-14T18:23:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:45 smithi097 ceph-mon[43486]: osdmap e66: 11 total, 10 up, 11 in 2023-12-14T18:23:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:45 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:45 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: osd.10 [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] boot 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: osdmap e67: 11 total, 11 up, 11 in 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:46 smithi093 ceph-mon[42070]: osdmap e68: 11 total, 11 up, 11 in 2023-12-14T18:23:46.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:23:46.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:23:46.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: osd.10 [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] boot 2023-12-14T18:23:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: osdmap e67: 11 total, 11 up, 11 in 2023-12-14T18:23:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:46 smithi097 ceph-mon[43486]: osdmap e68: 11 total, 11 up, 11 in 2023-12-14T18:23:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:23:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:23:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: from='client.24325 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi160:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:23:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: osd.10 [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] boot 2023-12-14T18:23:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: osdmap e67: 11 total, 11 up, 11 in 2023-12-14T18:23:46.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:23:46.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2023-12-14T18:23:46.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2023-12-14T18:23:46.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:46.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:46 smithi160 ceph-mon[43840]: osdmap e68: 11 total, 11 up, 11 in 2023-12-14T18:23:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:47 smithi093 ceph-mon[42070]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:47 smithi093 ceph-mon[42070]: osdmap e69: 11 total, 11 up, 11 in 2023-12-14T18:23:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:47 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/470537184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82c3b552-1386-4b96-89c6-4263d10a3f2c"}]: dispatch 2023-12-14T18:23:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:47 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82c3b552-1386-4b96-89c6-4263d10a3f2c"}]: dispatch 2023-12-14T18:23:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:47 smithi093 ceph-mon[42070]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82c3b552-1386-4b96-89c6-4263d10a3f2c"}]': finished 2023-12-14T18:23:47.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:47 smithi093 ceph-mon[42070]: osdmap e70: 12 total, 11 up, 12 in 2023-12-14T18:23:47.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:47 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:23:47.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:47 smithi097 ceph-mon[43486]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:47 smithi097 ceph-mon[43486]: osdmap e69: 11 total, 11 up, 11 in 2023-12-14T18:23:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:47 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/470537184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82c3b552-1386-4b96-89c6-4263d10a3f2c"}]: dispatch 2023-12-14T18:23:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:47 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82c3b552-1386-4b96-89c6-4263d10a3f2c"}]: dispatch 2023-12-14T18:23:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:47 smithi097 ceph-mon[43486]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82c3b552-1386-4b96-89c6-4263d10a3f2c"}]': finished 2023-12-14T18:23:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:47 smithi097 ceph-mon[43486]: osdmap e70: 12 total, 11 up, 12 in 2023-12-14T18:23:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:47 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:23:47.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:47 smithi160 ceph-mon[43840]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 59 MiB used, 894 GiB / 894 GiB avail 2023-12-14T18:23:47.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:47 smithi160 ceph-mon[43840]: osdmap e69: 11 total, 11 up, 11 in 2023-12-14T18:23:47.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:47 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/470537184' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82c3b552-1386-4b96-89c6-4263d10a3f2c"}]: dispatch 2023-12-14T18:23:47.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:47 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "82c3b552-1386-4b96-89c6-4263d10a3f2c"}]: dispatch 2023-12-14T18:23:47.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:47 smithi160 ceph-mon[43840]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "82c3b552-1386-4b96-89c6-4263d10a3f2c"}]': finished 2023-12-14T18:23:47.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:47 smithi160 ceph-mon[43840]: osdmap e70: 12 total, 11 up, 12 in 2023-12-14T18:23:47.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:47 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:23:48.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:48 smithi093 ceph-mon[42070]: from='client.? 172.21.15.160:0/3196760141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:48.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:48 smithi097 ceph-mon[43486]: from='client.? 172.21.15.160:0/3196760141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:48.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:48 smithi160 ceph-mon[43840]: from='client.? 172.21.15.160:0/3196760141' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2023-12-14T18:23:49.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:49 smithi093 ceph-mon[42070]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2023-12-14T18:23:49.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:49 smithi093 ceph-mon[42070]: osdmap e71: 12 total, 11 up, 12 in 2023-12-14T18:23:49.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:49 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:23:49.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:49 smithi097 ceph-mon[43486]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2023-12-14T18:23:49.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:49 smithi097 ceph-mon[43486]: osdmap e71: 12 total, 11 up, 12 in 2023-12-14T18:23:49.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:49 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:23:49.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:49 smithi160 ceph-mon[43840]: pgmap v186: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail; 148 KiB/s, 0 objects/s recovering 2023-12-14T18:23:49.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:49 smithi160 ceph-mon[43840]: osdmap e71: 12 total, 11 up, 12 in 2023-12-14T18:23:49.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:49 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:23:50.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:50 smithi097 ceph-mon[43486]: osdmap e72: 12 total, 11 up, 12 in 2023-12-14T18:23:50.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:50 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:23:50.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:50 smithi160 ceph-mon[43840]: osdmap e72: 12 total, 11 up, 12 in 2023-12-14T18:23:50.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:50 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:23:50.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:50 smithi093 ceph-mon[42070]: osdmap e72: 12 total, 11 up, 12 in 2023-12-14T18:23:50.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:50 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:23:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:51 smithi160 ceph-mon[43840]: pgmap v189: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:23:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:51 smithi097 ceph-mon[43486]: pgmap v189: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:23:51.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:51 smithi093 ceph-mon[42070]: pgmap v189: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 65 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2023-12-14T18:23:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:53 smithi160 ceph-mon[43840]: pgmap v190: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-14T18:23:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:53 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-14T18:23:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:53 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:53 smithi097 ceph-mon[43486]: pgmap v190: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-14T18:23:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:53 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-14T18:23:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:53 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:53.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:53 smithi093 ceph-mon[42070]: pgmap v190: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 111 KiB/s, 0 objects/s recovering 2023-12-14T18:23:53.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:53 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2023-12-14T18:23:53.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:53 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:54.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:54 smithi097 ceph-mon[43486]: Deploying daemon osd.11 on smithi160 2023-12-14T18:23:54.448 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:54 smithi160 ceph-mon[43840]: Deploying daemon osd.11 on smithi160 2023-12-14T18:23:54.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:54 smithi093 ceph-mon[42070]: Deploying daemon osd.11 on smithi160 2023-12-14T18:23:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:55 smithi160 ceph-mon[43840]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-12-14T18:23:55.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:55 smithi097 ceph-mon[43486]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-12-14T18:23:55.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:55 smithi093 ceph-mon[42070]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail; 82 KiB/s, 0 objects/s recovering 2023-12-14T18:23:56.744 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:56 smithi160 ceph-mon[43840]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:23:56.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:56 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:56.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:56 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:56.745 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:56 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:57.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:56 smithi093 ceph-mon[42070]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:23:57.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:56 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:57.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:56 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:57.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:56 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:57.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:56 smithi097 ceph-mon[43486]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:23:57.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:56 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:57.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:56 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:57.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:56 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:58.900 INFO:teuthology.orchestra.run.smithi160.stdout:Created osd(s) 11 on host 'smithi160' 2023-12-14T18:23:58.900 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.898+0000 7f2a337fe700 1 -- 172.21.15.160:0/2905209324 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f2a44061a10 con 0x7f2a2c0a1150 2023-12-14T18:23:58.902 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2a2c0a1150 msgr2=0x7f2a2c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:58.902 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2a2c0a1150 0x7f2a2c0a3600 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f2a38005950 tx=0x7f2a380058e0 comp rx=0 tx=0).stop 2023-12-14T18:23:58.902 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 msgr2=0x7f2a44196040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:23:58.902 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 0x7f2a44196040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f2a3400b5c0 tx=0x7f2a3400f690 comp rx=0 tx=0).stop 2023-12-14T18:23:58.902 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 shutdown_connections 2023-12-14T18:23:58.902 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2a2c0a1150 0x7f2a2c0a3600 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:58.902 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2a44100d70 0x7f2a4419d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:58.902 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2a440ff1e0 0x7f2a44196040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:58.903 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 --2- 172.21.15.160:0/2905209324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2a441003e0 0x7f2a44196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:23:58.903 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 >> 172.21.15.160:0/2905209324 conn(0x7f2a440fa7f0 msgr2=0x7f2a44103930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:23:58.903 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 shutdown_connections 2023-12-14T18:23:58.903 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:23:58.899+0000 7f2a48a22700 1 -- 172.21.15.160:0/2905209324 wait complete. 2023-12-14T18:23:59.271 DEBUG:teuthology.orchestra.run.smithi160:osd.11> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.11.service 2023-12-14T18:23:59.274 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2023-12-14T18:23:59.274 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd stat -f json 2023-12-14T18:23:59.306 INFO:journalctl@ceph.osd.11.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:23:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:23:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:23:59.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:23:59.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:59.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:23:59 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:23:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:23:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:23:59.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:59.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:23:59 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: pgmap v193: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:23:59.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:23:59.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:23:59.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:59.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:23:59.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:23:59.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:23:59.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:23:59 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:23:59.600 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:24:00.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.497+0000 7fe4d4029700 1 -- 172.21.15.93:0/3628178696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc103750 msgr2=0x7fe4cc105b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:00.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.497+0000 7fe4d4029700 1 --2- 172.21.15.93:0/3628178696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc103750 0x7fe4cc105b30 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fe4bc009990 tx=0x7fe4bc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.497+0000 7fe4d4029700 1 -- 172.21.15.93:0/3628178696 shutdown_connections 2023-12-14T18:24:00.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.497+0000 7fe4d4029700 1 --2- 172.21.15.93:0/3628178696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4cc106070 0x7fe4cc108450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.497+0000 7fe4d4029700 1 --2- 172.21.15.93:0/3628178696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc100e30 0x7fe4cc103210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.497+0000 7fe4d4029700 1 --2- 172.21.15.93:0/3628178696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc103750 0x7fe4cc105b30 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.497+0000 7fe4d4029700 1 -- 172.21.15.93:0/3628178696 >> 172.21.15.93:0/3628178696 conn(0x7fe4cc0fa7f0 msgr2=0x7fe4cc0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:00.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.497+0000 7fe4d4029700 1 -- 172.21.15.93:0/3628178696 shutdown_connections 2023-12-14T18:24:00.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.497+0000 7fe4d4029700 1 -- 172.21.15.93:0/3628178696 wait complete. 2023-12-14T18:24:00.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d4029700 1 Processor -- start 2023-12-14T18:24:00.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d4029700 1 -- start start 2023-12-14T18:24:00.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d4029700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc100e30 0x7fe4cc1960c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:00.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d4029700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4cc103750 0x7fe4cc196600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:00.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d4029700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc106070 0x7fe4cc19d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:00.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d4029700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc19dc50 con 0x7fe4cc106070 2023-12-14T18:24:00.503 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d4029700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc19dd90 con 0x7fe4cc100e30 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d4029700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4cc19df20 con 0x7fe4cc103750 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d25c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc106070 0x7fe4cc19d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d25c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc106070 0x7fe4cc19d680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44084/0 (socket says 172.21.15.93:44084) 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d25c6700 1 -- 172.21.15.93:0/1814982167 learned_addr learned my addr 172.21.15.93:0/1814982167 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d1dc5700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc100e30 0x7fe4cc1960c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d15c4700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4cc103750 0x7fe4cc196600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d25c6700 1 -- 172.21.15.93:0/1814982167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4cc103750 msgr2=0x7fe4cc196600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d25c6700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4cc103750 0x7fe4cc196600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d25c6700 1 -- 172.21.15.93:0/1814982167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc100e30 msgr2=0x7fe4cc1960c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:00.504 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d25c6700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc100e30 0x7fe4cc1960c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d25c6700 1 -- 172.21.15.93:0/1814982167 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4bc0095f0 con 0x7fe4cc106070 2023-12-14T18:24:00.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4d25c6700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc106070 0x7fe4cc19d680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4007f10 tx=0x7fe4c400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:00.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.498+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1814982167 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4017070 con 0x7fe4cc106070 2023-12-14T18:24:00.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.499+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe4cc10bcc0 con 0x7fe4cc106070 2023-12-14T18:24:00.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.499+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1814982167 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe4c400f040 con 0x7fe4cc106070 2023-12-14T18:24:00.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.499+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1814982167 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4c4005610 con 0x7fe4cc106070 2023-12-14T18:24:00.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.499+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4cc10bb30 con 0x7fe4cc106070 2023-12-14T18:24:00.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.500+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe4cc0fc360 con 0x7fe4cc106070 2023-12-14T18:24:00.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.500+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1814982167 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe4c4004ad0 con 0x7fe4cc106070 2023-12-14T18:24:00.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.501+0000 7fe4c2ffd700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe4b80a1150 0x7fe4b80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:00.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.501+0000 7fe4d1dc5700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe4b80a1150 0x7fe4b80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:00.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.501+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1814982167 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(72..72 src has 1..72) v4 ==== 6809+0+0 (secure 0 0 0) 0x7fe4c40cb9e0 con 0x7fe4cc106070 2023-12-14T18:24:00.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.501+0000 7fe4d1dc5700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe4b80a1150 0x7fe4b80a3600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8007c60 tx=0x7fe4c80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:00.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.505+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1814982167 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe4c4095710 con 0x7fe4cc106070 2023-12-14T18:24:00.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.696+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7fe4cc0618c0 con 0x7fe4cc106070 2023-12-14T18:24:00.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.697+0000 7fe4c2ffd700 1 -- 172.21.15.93:0/1814982167 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) v1 ==== 74+0+133 (secure 0 0 0) 0x7fe4c4098d30 con 0x7fe4cc106070 2023-12-14T18:24:00.699 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe4b80a1150 msgr2=0x7fe4b80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe4b80a1150 0x7fe4b80a3600 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fe4c8007c60 tx=0x7fe4c80073d0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc106070 msgr2=0x7fe4cc19d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc106070 0x7fe4cc19d680 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe4c4007f10 tx=0x7fe4c400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 shutdown_connections 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe4b80a1150 0x7fe4b80a3600 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe4cc106070 0x7fe4cc19d680 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe4cc100e30 0x7fe4cc1960c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 --2- 172.21.15.93:0/1814982167 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe4cc103750 0x7fe4cc196600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 >> 172.21.15.93:0/1814982167 conn(0x7fe4cc0fa7f0 msgr2=0x7fe4cc101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:00.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 shutdown_connections 2023-12-14T18:24:00.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:00.699+0000 7fe4d4029700 1 -- 172.21.15.93:0/1814982167 wait complete. 2023-12-14T18:24:00.746 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:24:00 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:24:00.486+0000 7fd6bf8e8640 -1 osd.11 0 log_to_monitors true 2023-12-14T18:24:01.069 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":72,"num_osds":12,"num_up_osds":11,"osd_up_since":1702578225,"num_in_osds":12,"osd_in_since":1702578226,"num_remapped_pgs":0} 2023-12-14T18:24:01.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:01 smithi093 ceph-mon[42070]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:24:01.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:01 smithi093 ceph-mon[42070]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-14T18:24:01.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:01 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1814982167' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:24:01.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:01 smithi160 ceph-mon[43840]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:24:01.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:01 smithi160 ceph-mon[43840]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-14T18:24:01.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:01 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1814982167' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:24:01.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:01 smithi097 ceph-mon[43486]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:24:01.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:01 smithi097 ceph-mon[43486]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2023-12-14T18:24:01.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:01 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1814982167' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:24:02.070 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd stat -f json 2023-12-14T18:24:02.388 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:24:02.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-14T18:24:02.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: osdmap e73: 12 total, 11 up, 12 in 2023-12-14T18:24:02.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: Detected new or changed devices on smithi160 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi160 to 4278M 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.482 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:24:02.483 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:24:02.483 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:24:02.483 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: osdmap e74: 12 total, 11 up, 12 in 2023-12-14T18:24:02.483 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:02 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:02.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:24:02.122+0000 7fd6b4f5f700 -1 osd.11 0 waiting for initial osdmap 2023-12-14T18:24:02.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:24:02.142+0000 7fd6afd52700 -1 osd.11 74 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-14T18:24:02.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-14T18:24:02.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: osdmap e73: 12 total, 11 up, 12 in 2023-12-14T18:24:02.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: Detected new or changed devices on smithi160 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi160 to 4278M 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:24:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:24:02.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:24:02.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: osdmap e74: 12 total, 11 up, 12 in 2023-12-14T18:24:02.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:02 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:02.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: osdmap e73: 12 total, 11 up, 12 in 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]: dispatch 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: Detected new or changed devices on smithi160 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi160 to 4278M 2023-12-14T18:24:02.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:02.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:02.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:24:02.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:02.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:24:02.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:24:02.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238]' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi160", "root=default"]}]': finished 2023-12-14T18:24:02.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: osdmap e74: 12 total, 11 up, 12 in 2023-12-14T18:24:02.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:02 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:03.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.373+0000 7f62cb9e6700 1 -- 172.21.15.93:0/1707381650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c40ff420 msgr2=0x7f62c40ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.373+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/1707381650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c40ff420 0x7f62c40ff830 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f62b8009990 tx=0x7f62b8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.373+0000 7f62cb9e6700 1 -- 172.21.15.93:0/1707381650 shutdown_connections 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.373+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/1707381650 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c40ff420 0x7f62c40ff830 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.373+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/1707381650 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62c4100fb0 0x7f62c4107870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.373+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/1707381650 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62c4100620 0x7f62c4100a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.373+0000 7f62cb9e6700 1 -- 172.21.15.93:0/1707381650 >> 172.21.15.93:0/1707381650 conn(0x7f62c40faa50 msgr2=0x7f62c40fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.373+0000 7f62cb9e6700 1 -- 172.21.15.93:0/1707381650 shutdown_connections 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.373+0000 7f62cb9e6700 1 -- 172.21.15.93:0/1707381650 wait complete. 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62cb9e6700 1 Processor -- start 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62cb9e6700 1 -- start start 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62cb9e6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62c40ff420 0x7f62c4069c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62cb9e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62c4100620 0x7f62c419b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:03.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62cb9e6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c4100fb0 0x7f62c419d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62cb9e6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c419de40 con 0x7f62c4100fb0 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62cb9e6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c419dfb0 con 0x7f62c40ff420 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62cb9e6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62c419e170 con 0x7f62c4100620 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9f83700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c4100fb0 0x7f62c419d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9f83700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c4100fb0 0x7f62c419d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36004/0 (socket says 172.21.15.93:36004) 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9f83700 1 -- 172.21.15.93:0/2629605587 learned_addr learned my addr 172.21.15.93:0/2629605587 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9782700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62c40ff420 0x7f62c4069c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c8f81700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62c4100620 0x7f62c419b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9f83700 1 -- 172.21.15.93:0/2629605587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62c4100620 msgr2=0x7f62c419b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9f83700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62c4100620 0x7f62c419b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9f83700 1 -- 172.21.15.93:0/2629605587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62c40ff420 msgr2=0x7f62c4069c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9f83700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62c40ff420 0x7f62c4069c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9f83700 1 -- 172.21.15.93:0/2629605587 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62b80095f0 con 0x7f62c4100fb0 2023-12-14T18:24:03.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.374+0000 7f62c9f83700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c4100fb0 0x7f62c419d6a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00b970 tx=0x7f62bc00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:03.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.375+0000 7f62b67fc700 1 -- 172.21.15.93:0/2629605587 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc00c930 con 0x7f62c4100fb0 2023-12-14T18:24:03.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.375+0000 7f62b67fc700 1 -- 172.21.15.93:0/2629605587 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f62bc014440 con 0x7f62c4100fb0 2023-12-14T18:24:03.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.375+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62c410ae70 con 0x7f62c4100fb0 2023-12-14T18:24:03.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.375+0000 7f62b67fc700 1 -- 172.21.15.93:0/2629605587 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62bc0075d0 con 0x7f62c4100fb0 2023-12-14T18:24:03.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.375+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62c410ace0 con 0x7f62c4100fb0 2023-12-14T18:24:03.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.375+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62c41069c0 con 0x7f62c4100fb0 2023-12-14T18:24:03.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.377+0000 7f62b67fc700 1 -- 172.21.15.93:0/2629605587 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f62bc0145b0 con 0x7f62c4100fb0 2023-12-14T18:24:03.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.378+0000 7f62b67fc700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f62b00a1150 0x7f62b00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:03.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.378+0000 7f62c9782700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f62b00a1150 0x7f62b00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:03.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.378+0000 7f62b67fc700 1 -- 172.21.15.93:0/2629605587 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f62bc006e40 con 0x7f62c4100fb0 2023-12-14T18:24:03.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.378+0000 7f62c9782700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f62b00a1150 0x7f62b00a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f62b8006010 tx=0x7f62b8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:03.383 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.381+0000 7f62b67fc700 1 -- 172.21.15.93:0/2629605587 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f62bc095ba0 con 0x7f62c4100fb0 2023-12-14T18:24:03.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:03 smithi160 ceph-mon[43840]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:24:03.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:03 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:03.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:03 smithi093 ceph-mon[42070]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:24:03.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:03 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.573+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f62c402cfe0 con 0x7f62c4100fb0 2023-12-14T18:24:03.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.573+0000 7f62b67fc700 1 -- 172.21.15.93:0/2629605587 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v75) v1 ==== 74+0+133 (secure 0 0 0) 0x7f62bc0991c0 con 0x7f62c4100fb0 2023-12-14T18:24:03.576 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f62b00a1150 msgr2=0x7f62b00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f62b00a1150 0x7f62b00a3600 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f62b8006010 tx=0x7f62b8005960 comp rx=0 tx=0).stop 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c4100fb0 msgr2=0x7f62c419d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c4100fb0 0x7f62c419d6a0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f62bc00b970 tx=0x7f62bc00bc80 comp rx=0 tx=0).stop 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 shutdown_connections 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f62b00a1150 0x7f62b00a3600 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f62c4100fb0 0x7f62c419d6a0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f62c40ff420 0x7f62c4069c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 --2- 172.21.15.93:0/2629605587 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f62c4100620 0x7f62c419b050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 >> 172.21.15.93:0/2629605587 conn(0x7f62c40faa50 msgr2=0x7f62c4103b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 shutdown_connections 2023-12-14T18:24:03.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:03.576+0000 7f62cb9e6700 1 -- 172.21.15.93:0/2629605587 wait complete. 2023-12-14T18:24:03.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:03 smithi097 ceph-mon[43486]: pgmap v196: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:24:03.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:03 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:03.959 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":75,"num_osds":12,"num_up_osds":12,"osd_up_since":1702578243,"num_in_osds":12,"osd_in_since":1702578226,"num_remapped_pgs":0} 2023-12-14T18:24:03.960 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd dump --format=json 2023-12-14T18:24:04.271 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:24:04.389 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:04 smithi093 ceph-mon[42070]: purged_snaps scrub starts 2023-12-14T18:24:04.389 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:04 smithi093 ceph-mon[42070]: purged_snaps scrub ok 2023-12-14T18:24:04.389 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:04.389 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:04 smithi093 ceph-mon[42070]: osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] boot 2023-12-14T18:24:04.389 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:04 smithi093 ceph-mon[42070]: osdmap e75: 12 total, 12 up, 12 in 2023-12-14T18:24:04.389 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:04 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:04.389 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:04 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2629605587' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:24:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:04 smithi160 ceph-mon[43840]: purged_snaps scrub starts 2023-12-14T18:24:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:04 smithi160 ceph-mon[43840]: purged_snaps scrub ok 2023-12-14T18:24:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:04 smithi160 ceph-mon[43840]: osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] boot 2023-12-14T18:24:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:04 smithi160 ceph-mon[43840]: osdmap e75: 12 total, 12 up, 12 in 2023-12-14T18:24:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:04 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:04 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2629605587' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:24:04.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:04 smithi097 ceph-mon[43486]: purged_snaps scrub starts 2023-12-14T18:24:04.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:04 smithi097 ceph-mon[43486]: purged_snaps scrub ok 2023-12-14T18:24:04.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:04.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:04 smithi097 ceph-mon[43486]: osd.11 [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] boot 2023-12-14T18:24:04.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:04 smithi097 ceph-mon[43486]: osdmap e75: 12 total, 12 up, 12 in 2023-12-14T18:24:04.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:04 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:24:04.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:04 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2629605587' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2023-12-14T18:24:05.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.306+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/289420788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04100560 msgr2=0x7f2b04100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:05.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.306+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/289420788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04100560 0x7f2b04100930 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2af80099c0 tx=0x7f2af8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.306+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/289420788 shutdown_connections 2023-12-14T18:24:05.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.306+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/289420788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04100560 0x7f2b04100930 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.306+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/289420788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b04105600 0x7f2b0410d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.306+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/289420788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b04100f00 0x7f2b04104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.306+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/289420788 >> 172.21.15.93:0/289420788 conn(0x7f2b040fbde0 msgr2=0x7f2b040fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:05.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.306+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/289420788 shutdown_connections 2023-12-14T18:24:05.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.306+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/289420788 wait complete. 2023-12-14T18:24:05.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0c0ed700 1 Processor -- start 2023-12-14T18:24:05.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0c0ed700 1 -- start start 2023-12-14T18:24:05.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0c0ed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b04100560 0x7f2b04196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:05.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0c0ed700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b04100f00 0x7f2b041969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:05.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0c0ed700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04105600 0x7f2b0419ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0c0ed700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0419b420 con 0x7f2b04105600 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0c0ed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0419b590 con 0x7f2b04100560 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0c0ed700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b0419b750 con 0x7f2b04100f00 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0a68a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04105600 0x7f2b0419ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.307+0000 7f2b0a68a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04105600 0x7f2b0419ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36028/0 (socket says 172.21.15.93:36028) 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b0a68a700 1 -- 172.21.15.93:0/4174302815 learned_addr learned my addr 172.21.15.93:0/4174302815 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b09688700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b04100f00 0x7f2b041969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b09e89700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b04100560 0x7f2b04196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b0a68a700 1 -- 172.21.15.93:0/4174302815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b04100f00 msgr2=0x7f2b041969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:05.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b0a68a700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b04100f00 0x7f2b041969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b0a68a700 1 -- 172.21.15.93:0/4174302815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b04100560 msgr2=0x7f2b04196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:05.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b0a68a700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b04100560 0x7f2b04196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b0a68a700 1 -- 172.21.15.93:0/4174302815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2af80095f0 con 0x7f2b04105600 2023-12-14T18:24:05.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b09688700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b04100f00 0x7f2b041969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:05.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b0a68a700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04105600 0x7f2b0419ae10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b970 tx=0x7f2afc00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:05.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2af6ffd700 1 -- 172.21.15.93:0/4174302815 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc00c9b0 con 0x7f2b04105600 2023-12-14T18:24:05.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2af6ffd700 1 -- 172.21.15.93:0/4174302815 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2afc014440 con 0x7f2b04105600 2023-12-14T18:24:05.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2af6ffd700 1 -- 172.21.15.93:0/4174302815 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2afc0075d0 con 0x7f2b04105600 2023-12-14T18:24:05.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b04110430 con 0x7f2b04105600 2023-12-14T18:24:05.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.308+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b041102a0 con 0x7f2b04105600 2023-12-14T18:24:05.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.309+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b041105c0 con 0x7f2b04105600 2023-12-14T18:24:05.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.310+0000 7f2af6ffd700 1 -- 172.21.15.93:0/4174302815 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2afc0145b0 con 0x7f2b04105600 2023-12-14T18:24:05.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.311+0000 7f2af6ffd700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2af00a1150 0x7f2af00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:05.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.311+0000 7f2b09e89700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2af00a1150 0x7f2af00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:05.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.311+0000 7f2af6ffd700 1 -- 172.21.15.93:0/4174302815 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2afc006e40 con 0x7f2b04105600 2023-12-14T18:24:05.316 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.311+0000 7f2b09e89700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2af00a1150 0x7f2af00a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000c00 tx=0x7f2af8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:05.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.314+0000 7f2af6ffd700 1 -- 172.21.15.93:0/4174302815 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2afc095c70 con 0x7f2b04105600 2023-12-14T18:24:05.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:05 smithi160 ceph-mon[43840]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:24:05.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:05 smithi160 ceph-mon[43840]: osdmap e76: 12 total, 12 up, 12 in 2023-12-14T18:24:05.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.505+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b04066ea0 con 0x7f2b04105600 2023-12-14T18:24:05.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.506+0000 7f2af6ffd700 1 -- 172.21.15.93:0/4174302815 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f2afc099290 con 0x7f2b04105600 2023-12-14T18:24:05.509 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:05.509 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":76,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:04.133253+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:05.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2af00a1150 msgr2=0x7f2af00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:05.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2af00a1150 0x7f2af00a3600 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f2af8000c00 tx=0x7f2af8005fb0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04105600 msgr2=0x7f2b0419ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:05.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04105600 0x7f2b0419ae10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2afc00b970 tx=0x7f2afc00bd30 comp rx=0 tx=0).stop 2023-12-14T18:24:05.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 shutdown_connections 2023-12-14T18:24:05.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2af00a1150 0x7f2af00a3600 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b04105600 0x7f2b0419ae10 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b04100560 0x7f2b04196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 --2- 172.21.15.93:0/4174302815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b04100f00 0x7f2b041969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:05.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 >> 172.21.15.93:0/4174302815 conn(0x7f2b040fbde0 msgr2=0x7f2b040fc7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:05.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 shutdown_connections 2023-12-14T18:24:05.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:05.509+0000 7f2b0c0ed700 1 -- 172.21.15.93:0/4174302815 wait complete. 2023-12-14T18:24:05.535 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:05 smithi093 ceph-mon[42070]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:24:05.535 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:05 smithi093 ceph-mon[42070]: osdmap e76: 12 total, 12 up, 12 in 2023-12-14T18:24:05.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:05 smithi097 ceph-mon[43486]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 66 MiB used, 983 GiB / 983 GiB avail 2023-12-14T18:24:05.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:05 smithi097 ceph-mon[43486]: osdmap e76: 12 total, 12 up, 12 in 2023-12-14T18:24:05.896 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-14T18:24:05.896 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd pool get .mgr pg_num 2023-12-14T18:24:06.205 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:24:06.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:06 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4174302815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:06.500 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:06 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4174302815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:06.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:06 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4174302815' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:07.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.275+0000 7f4f19718700 1 -- 172.21.15.93:0/627140157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 msgr2=0x7f4f1410d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:07.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.275+0000 7f4f19718700 1 --2- 172.21.15.93:0/627140157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 0x7f4f1410d170 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c00b0d0 tx=0x7f4f0c00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.275+0000 7f4f19718700 1 -- 172.21.15.93:0/627140157 shutdown_connections 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.275+0000 7f4f19718700 1 --2- 172.21.15.93:0/627140157 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 0x7f4f1410d170 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.275+0000 7f4f19718700 1 --2- 172.21.15.93:0/627140157 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f14100550 0x7f4f14100920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.275+0000 7f4f19718700 1 --2- 172.21.15.93:0/627140157 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f14100ef0 0x7f4f14104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.275+0000 7f4f19718700 1 -- 172.21.15.93:0/627140157 >> 172.21.15.93:0/627140157 conn(0x7f4f140fbdf0 msgr2=0x7f4f140fe200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.275+0000 7f4f19718700 1 -- 172.21.15.93:0/627140157 shutdown_connections 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.275+0000 7f4f19718700 1 -- 172.21.15.93:0/627140157 wait complete. 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f19718700 1 Processor -- start 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f19718700 1 -- start start 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f19718700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f14100550 0x7f4f14196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:07.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f19718700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f14100ef0 0x7f4f14196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f19718700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 0x7f4f1419ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f19718700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419b410 con 0x7f4f141055f0 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f19718700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419b580 con 0x7f4f14100ef0 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f19718700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f1419b740 con 0x7f4f14100550 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f137fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 0x7f4f1419ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f137fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 0x7f4f1419ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36036/0 (socket says 172.21.15.93:36036) 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f137fe700 1 -- 172.21.15.93:0/3893289534 learned_addr learned my addr 172.21.15.93:0/3893289534 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f12ffd700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f14100550 0x7f4f14196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.276+0000 7f4f127fc700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f14100ef0 0x7f4f14196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4f137fe700 1 -- 172.21.15.93:0/3893289534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f14100550 msgr2=0x7f4f14196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:07.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4f137fe700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f14100550 0x7f4f14196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4f137fe700 1 -- 172.21.15.93:0/3893289534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f14100ef0 msgr2=0x7f4f14196990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:07.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4f137fe700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f14100ef0 0x7f4f14196990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4f137fe700 1 -- 172.21.15.93:0/3893289534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f0c009d00 con 0x7f4f141055f0 2023-12-14T18:24:07.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4f137fe700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 0x7f4f1419ae00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c007870 tx=0x7f4f0c0078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:07.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4efbfff700 1 -- 172.21.15.93:0/3893289534 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c00e040 con 0x7f4f141055f0 2023-12-14T18:24:07.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f14110330 con 0x7f4f141055f0 2023-12-14T18:24:07.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4efbfff700 1 -- 172.21.15.93:0/3893289534 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4f0c003e10 con 0x7f4f141055f0 2023-12-14T18:24:07.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4efbfff700 1 -- 172.21.15.93:0/3893289534 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f0c01ba10 con 0x7f4f141055f0 2023-12-14T18:24:07.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.277+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f141101a0 con 0x7f4f141055f0 2023-12-14T18:24:07.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.278+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f141104c0 con 0x7f4f141055f0 2023-12-14T18:24:07.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.279+0000 7f4efbfff700 1 -- 172.21.15.93:0/3893289534 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4f0c024400 con 0x7f4f141055f0 2023-12-14T18:24:07.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.280+0000 7f4efbfff700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4efc0a1150 0x7f4efc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:07.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.280+0000 7f4f12ffd700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4efc0a1150 0x7f4efc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:07.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.280+0000 7f4efbfff700 1 -- 172.21.15.93:0/3893289534 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f4f0c0cdc00 con 0x7f4f141055f0 2023-12-14T18:24:07.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.280+0000 7f4f12ffd700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4efc0a1150 0x7f4efc0a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f08005950 tx=0x7f4f0800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:07.285 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.284+0000 7f4efbfff700 1 -- 172.21.15.93:0/3893289534 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4f0c0977e0 con 0x7f4f141055f0 2023-12-14T18:24:07.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.470+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4f141978b0 con 0x7f4f141055f0 2023-12-14T18:24:07.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.471+0000 7f4efbfff700 1 -- 172.21.15.93:0/3893289534 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4f0c09ae00 con 0x7f4f141055f0 2023-12-14T18:24:07.473 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:24:07.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4efc0a1150 msgr2=0x7f4efc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:07.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4efc0a1150 0x7f4efc0a3600 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4f08005950 tx=0x7f4f0800a300 comp rx=0 tx=0).stop 2023-12-14T18:24:07.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 msgr2=0x7f4f1419ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:07.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 0x7f4f1419ae00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4f0c007870 tx=0x7f4f0c0078a0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 shutdown_connections 2023-12-14T18:24:07.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4efc0a1150 0x7f4efc0a3600 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4f141055f0 0x7f4f1419ae00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4f14100550 0x7f4f14196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 --2- 172.21.15.93:0/3893289534 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4f14100ef0 0x7f4f14196990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:07.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 >> 172.21.15.93:0/3893289534 conn(0x7f4f140fbdf0 msgr2=0x7f4f140fc770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:07.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 shutdown_connections 2023-12-14T18:24:07.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:07.473+0000 7f4f19718700 1 -- 172.21.15.93:0/3893289534 wait complete. 2023-12-14T18:24:07.495 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:07 smithi093 ceph-mon[42070]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:07.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:07 smithi160 ceph-mon[43840]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:07.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:07 smithi097 ceph-mon[43486]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:07.859 INFO:tasks.cephadm:Adding mds.a on smithi093 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.d on smithi093 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.g on smithi093 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.j on smithi093 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.b on smithi097 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.e on smithi097 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.h on smithi097 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.k on smithi097 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.c on smithi160 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.f on smithi160 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.i on smithi160 2023-12-14T18:24:07.860 INFO:tasks.cephadm:Adding mds.l on smithi160 2023-12-14T18:24:07.861 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph orch apply mds all '12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi160=c;smithi160=f;smithi160=i;smithi160=l' 2023-12-14T18:24:08.185 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:24:08.443 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:08 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3893289534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:08.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:08 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3893289534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:08.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:08 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3893289534' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:09.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.178+0000 7f58a57cd700 1 -- 172.21.15.160:0/2224171694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 msgr2=0x7f58a00fed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:09.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.178+0000 7f58a57cd700 1 --2- 172.21.15.160:0/2224171694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 0x7f58a00fed60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5894009990 tx=0x7f5894009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.178+0000 7f58a57cd700 1 -- 172.21.15.160:0/2224171694 shutdown_connections 2023-12-14T18:24:09.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.178+0000 7f58a57cd700 1 --2- 172.21.15.160:0/2224171694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a01004e0 0x7f58a0106da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.178+0000 7f58a57cd700 1 --2- 172.21.15.160:0/2224171694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 0x7f58a00fed60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.183 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.178+0000 7f58a57cd700 1 --2- 172.21.15.160:0/2224171694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58a00ffb50 0x7f58a00fffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.178+0000 7f58a57cd700 1 -- 172.21.15.160:0/2224171694 >> 172.21.15.160:0/2224171694 conn(0x7f58a00f9f80 msgr2=0x7f58a00fc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.178+0000 7f58a57cd700 1 -- 172.21.15.160:0/2224171694 shutdown_connections 2023-12-14T18:24:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.178+0000 7f58a57cd700 1 -- 172.21.15.160:0/2224171694 wait complete. 2023-12-14T18:24:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f58a57cd700 1 Processor -- start 2023-12-14T18:24:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f58a57cd700 1 -- start start 2023-12-14T18:24:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f58a57cd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 0x7f58a0194df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f58a57cd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a00ffb50 0x7f58a0195330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:09.184 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f58a57cd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58a01004e0 0x7f58a019c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:09.185 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f58a57cd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a019c980 con 0x7f58a01004e0 2023-12-14T18:24:09.185 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f58a57cd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a019cac0 con 0x7f58a00ffb50 2023-12-14T18:24:09.185 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f58a57cd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58a019cc50 con 0x7f58a00fe950 2023-12-14T18:24:09.185 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f589effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 0x7f58a0194df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:09.185 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f589effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 0x7f58a0194df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.160:33038/0 (socket says 172.21.15.160:33038) 2023-12-14T18:24:09.185 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.179+0000 7f589effd700 1 -- 172.21.15.160:0/1602283020 learned_addr learned my addr 172.21.15.160:0/1602283020 (peer_addr_for_me v2:172.21.15.160:0/0) 2023-12-14T18:24:09.185 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f589f7fe700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58a01004e0 0x7f58a019c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:09.185 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f589e7fc700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a00ffb50 0x7f58a0195330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:09.186 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f589effd700 1 -- 172.21.15.160:0/1602283020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a00ffb50 msgr2=0x7f58a0195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:09.186 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f589effd700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a00ffb50 0x7f58a0195330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.186 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f589effd700 1 -- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58a01004e0 msgr2=0x7f58a019c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:09.186 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f589effd700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58a01004e0 0x7f58a019c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.186 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f589effd700 1 -- 172.21.15.160:0/1602283020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58940095f0 con 0x7f58a00fe950 2023-12-14T18:24:09.186 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f589e7fc700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a00ffb50 0x7f58a0195330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:24:09.186 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f589effd700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 0x7f58a0194df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5894005910 tx=0x7f589400f6e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:09.186 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f5887fff700 1 -- 172.21.15.160:0/1602283020 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f589401d070 con 0x7f58a00fe950 2023-12-14T18:24:09.186 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58a010a2a0 con 0x7f58a00fe950 2023-12-14T18:24:09.187 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f5887fff700 1 -- 172.21.15.160:0/1602283020 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5894022470 con 0x7f58a00fe950 2023-12-14T18:24:09.187 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f5887fff700 1 -- 172.21.15.160:0/1602283020 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5894017650 con 0x7f58a00fe950 2023-12-14T18:24:09.187 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.180+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58a010a110 con 0x7f58a00fe950 2023-12-14T18:24:09.187 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.181+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58a01064d0 con 0x7f58a00fe950 2023-12-14T18:24:09.188 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.182+0000 7f5887fff700 1 -- 172.21.15.160:0/1602283020 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f58940225e0 con 0x7f58a00fe950 2023-12-14T18:24:09.188 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.182+0000 7f5887fff700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58880a1150 0x7f58880a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:09.189 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.183+0000 7f5887fff700 1 -- 172.21.15.160:0/1602283020 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f58940cd9e0 con 0x7f58a00fe950 2023-12-14T18:24:09.189 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.183+0000 7f589e7fc700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58880a1150 0x7f58880a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:09.189 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.183+0000 7f589e7fc700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58880a1150 0x7f58880a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5890005fd0 tx=0x7f5890005e20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:09.189 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.186+0000 7f5887fff700 1 -- 172.21.15.160:0/1602283020 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5894097660 con 0x7f58a00fe950 2023-12-14T18:24:09.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:09 smithi160 ceph-mon[43840]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:09.386 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.385+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi160=c;smithi160=f;smithi160=i;smithi160=l", "target": ["mon-mgr", ""]}) v1 -- 0x7f58a0061c00 con 0x7f58880a1150 2023-12-14T18:24:09.394 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.393+0000 7f5887fff700 1 -- 172.21.15.160:0/1602283020 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f58a0061c00 con 0x7f58880a1150 2023-12-14T18:24:09.395 INFO:teuthology.orchestra.run.smithi160.stdout:Scheduled mds.all update... 2023-12-14T18:24:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58880a1150 msgr2=0x7f58880a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58880a1150 0x7f58880a3600 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f5890005fd0 tx=0x7f5890005e20 comp rx=0 tx=0).stop 2023-12-14T18:24:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 msgr2=0x7f58a0194df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 0x7f58a0194df0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5894005910 tx=0x7f589400f6e0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 shutdown_connections 2023-12-14T18:24:09.397 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58880a1150 0x7f58880a3600 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58a01004e0 0x7f58a019c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58a00fe950 0x7f58a0194df0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 --2- 172.21.15.160:0/1602283020 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58a00ffb50 0x7f58a0195330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 >> 172.21.15.160:0/1602283020 conn(0x7f58a00f9f80 msgr2=0x7f58a0103610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 shutdown_connections 2023-12-14T18:24:09.398 INFO:teuthology.orchestra.run.smithi160.stderr:2023-12-14T18:24:09.395+0000 7f58a57cd700 1 -- 172.21.15.160:0/1602283020 wait complete. 2023-12-14T18:24:09.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:09 smithi093 ceph-mon[42070]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:09.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:09 smithi097 ceph-mon[43486]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 72 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:09.721 DEBUG:teuthology.orchestra.run.smithi093:mds.a> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.a.service 2023-12-14T18:24:09.724 DEBUG:teuthology.orchestra.run.smithi093:mds.d> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.d.service 2023-12-14T18:24:09.753 INFO:journalctl@ceph.mds.a.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:24:09.754 DEBUG:teuthology.orchestra.run.smithi093:mds.g> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.g.service 2023-12-14T18:24:09.780 INFO:journalctl@ceph.mds.d.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:24:09.782 DEBUG:teuthology.orchestra.run.smithi093:mds.j> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.j.service 2023-12-14T18:24:09.807 INFO:journalctl@ceph.mds.g.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:24:09.808 DEBUG:teuthology.orchestra.run.smithi097:mds.b> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.b.service 2023-12-14T18:24:09.812 DEBUG:teuthology.orchestra.run.smithi097:mds.e> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.e.service 2023-12-14T18:24:09.835 INFO:journalctl@ceph.mds.j.smithi093.stdout:-- Logs begin at Thu 2023-12-14 18:05:32 UTC. -- 2023-12-14T18:24:09.851 INFO:journalctl@ceph.mds.b.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:24:09.852 DEBUG:teuthology.orchestra.run.smithi097:mds.h> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.h.service 2023-12-14T18:24:09.882 INFO:journalctl@ceph.mds.e.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:24:09.884 DEBUG:teuthology.orchestra.run.smithi097:mds.k> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.k.service 2023-12-14T18:24:09.911 INFO:journalctl@ceph.mds.h.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:24:09.912 DEBUG:teuthology.orchestra.run.smithi160:mds.c> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.c.service 2023-12-14T18:24:09.915 DEBUG:teuthology.orchestra.run.smithi160:mds.f> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.f.service 2023-12-14T18:24:09.938 INFO:journalctl@ceph.mds.k.smithi097.stdout:-- Logs begin at Thu 2023-12-14 18:04:57 UTC. -- 2023-12-14T18:24:09.944 INFO:journalctl@ceph.mds.c.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:24:09.945 DEBUG:teuthology.orchestra.run.smithi160:mds.i> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.i.service 2023-12-14T18:24:09.973 INFO:journalctl@ceph.mds.f.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:24:09.974 DEBUG:teuthology.orchestra.run.smithi160:mds.l> sudo journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.l.service 2023-12-14T18:24:10.000 INFO:journalctl@ceph.mds.i.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:24:10.001 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2023-12-14T18:24:10.002 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:24:10.027 INFO:journalctl@ceph.mds.l.smithi160.stdout:-- Logs begin at Thu 2023-12-14 18:05:50 UTC. -- 2023-12-14T18:24:10.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.138+0000 7f06527a7700 1 -- 172.21.15.93:0/1539183580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 msgr2=0x7f064c101920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.138+0000 7f06527a7700 1 --2- 172.21.15.93:0/1539183580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 0x7f064c101920 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0640009990 tx=0x7f0640009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.138+0000 7f06527a7700 1 -- 172.21.15.93:0/1539183580 shutdown_connections 2023-12-14T18:24:10.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.138+0000 7f06527a7700 1 --2- 172.21.15.93:0/1539183580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f064c101e60 0x7f064c108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.138+0000 7f06527a7700 1 --2- 172.21.15.93:0/1539183580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064c0fe500 0x7f064c0fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.138+0000 7f06527a7700 1 --2- 172.21.15.93:0/1539183580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 0x7f064c101920 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.138+0000 7f06527a7700 1 -- 172.21.15.93:0/1539183580 >> 172.21.15.93:0/1539183580 conn(0x7f064c0f9ea0 msgr2=0x7f064c0fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.138+0000 7f06527a7700 1 -- 172.21.15.93:0/1539183580 shutdown_connections 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.138+0000 7f06527a7700 1 -- 172.21.15.93:0/1539183580 wait complete. 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f06527a7700 1 Processor -- start 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f06527a7700 1 -- start start 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f06527a7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064c0fe500 0x7f064c100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f06527a7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 0x7f064c0ff480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f06527a7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f064c101e60 0x7f064c0ff9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f06527a7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c100160 con 0x7f064c1014d0 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f06527a7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c1002d0 con 0x7f064c0fe500 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f06527a7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f064c1a0010 con 0x7f064c101e60 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f064b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 0x7f064c0ff480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f064b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 0x7f064c0ff480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36068/0 (socket says 172.21.15.93:36068) 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f064b7fe700 1 -- 172.21.15.93:0/1227386039 learned_addr learned my addr 172.21.15.93:0/1227386039 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f064bfff700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064c0fe500 0x7f064c100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f0650d44700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f064c101e60 0x7f064c0ff9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f064b7fe700 1 -- 172.21.15.93:0/1227386039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f064c101e60 msgr2=0x7f064c0ff9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f064b7fe700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f064c101e60 0x7f064c0ff9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f064b7fe700 1 -- 172.21.15.93:0/1227386039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064c0fe500 msgr2=0x7f064c100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f064b7fe700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064c0fe500 0x7f064c100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.139+0000 7f064b7fe700 1 -- 172.21.15.93:0/1227386039 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06400095f0 con 0x7f064c1014d0 2023-12-14T18:24:10.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.140+0000 7f064b7fe700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 0x7f064c0ff480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0640000c00 tx=0x7f064000f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:10.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.140+0000 7f06497fa700 1 -- 172.21.15.93:0/1227386039 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f064001d070 con 0x7f064c1014d0 2023-12-14T18:24:10.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.140+0000 7f06497fa700 1 -- 172.21.15.93:0/1227386039 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f064000fb50 con 0x7f064c1014d0 2023-12-14T18:24:10.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.140+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f064c10bba0 con 0x7f064c1014d0 2023-12-14T18:24:10.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.140+0000 7f06497fa700 1 -- 172.21.15.93:0/1227386039 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06400173f0 con 0x7f064c1014d0 2023-12-14T18:24:10.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.140+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f064c10ba10 con 0x7f064c1014d0 2023-12-14T18:24:10.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.141+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f064c1a0640 con 0x7f064c1014d0 2023-12-14T18:24:10.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.142+0000 7f06497fa700 1 -- 172.21.15.93:0/1227386039 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f064000fcc0 con 0x7f064c1014d0 2023-12-14T18:24:10.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.142+0000 7f06497fa700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f06340a1150 0x7f06340a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.142+0000 7f064bfff700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f06340a1150 0x7f06340a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.142+0000 7f06497fa700 1 -- 172.21.15.93:0/1227386039 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f06400cdae0 con 0x7f064c1014d0 2023-12-14T18:24:10.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.143+0000 7f064bfff700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f06340a1150 0x7f06340a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f064c068e90 tx=0x7f063c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:10.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.146+0000 7f06497fa700 1 -- 172.21.15.93:0/1227386039 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0640097760 con 0x7f064c1014d0 2023-12-14T18:24:10.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.337+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f064c107220 con 0x7f064c1014d0 2023-12-14T18:24:10.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.338+0000 7f06497fa700 1 -- 172.21.15.93:0/1227386039 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18350 (secure 0 0 0) 0x7f064009ad80 con 0x7f064c1014d0 2023-12-14T18:24:10.340 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:10.340 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":76,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:04.133253+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:10.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f06340a1150 msgr2=0x7f06340a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f06340a1150 0x7f06340a3600 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f064c068e90 tx=0x7f063c00b410 comp rx=0 tx=0).stop 2023-12-14T18:24:10.342 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 msgr2=0x7f064c0ff480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 0x7f064c0ff480 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0640000c00 tx=0x7f064000f710 comp rx=0 tx=0).stop 2023-12-14T18:24:10.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 shutdown_connections 2023-12-14T18:24:10.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f06340a1150 0x7f06340a3600 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f064c101e60 0x7f064c0ff9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f064c0fe500 0x7f064c100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 --2- 172.21.15.93:0/1227386039 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f064c1014d0 0x7f064c0ff480 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 >> 172.21.15.93:0/1227386039 conn(0x7f064c0f9ea0 msgr2=0x7f064c0fc190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:10.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 shutdown_connections 2023-12-14T18:24:10.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.340+0000 7f06527a7700 1 -- 172.21.15.93:0/1227386039 wait complete. 2023-12-14T18:24:10.355 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-14T18:24:10.355 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:24:10.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.478+0000 7f8b97421700 1 -- 172.21.15.93:0/1298279218 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 msgr2=0x7f8b90103890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.478+0000 7f8b97421700 1 --2- 172.21.15.93:0/1298279218 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 0x7f8b90103890 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f8b80009990 tx=0x7f8b80009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.478+0000 7f8b97421700 1 -- 172.21.15.93:0/1298279218 shutdown_connections 2023-12-14T18:24:10.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.478+0000 7f8b97421700 1 --2- 172.21.15.93:0/1298279218 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b90103dd0 0x7f8b90104240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.478+0000 7f8b97421700 1 --2- 172.21.15.93:0/1298279218 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b900feab0 0x7f8b90100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.478+0000 7f8b97421700 1 --2- 172.21.15.93:0/1298279218 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 0x7f8b90103890 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.478+0000 7f8b97421700 1 -- 172.21.15.93:0/1298279218 >> 172.21.15.93:0/1298279218 conn(0x7f8b900fa740 msgr2=0x7f8b900fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:10.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.478+0000 7f8b97421700 1 -- 172.21.15.93:0/1298279218 shutdown_connections 2023-12-14T18:24:10.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.478+0000 7f8b97421700 1 -- 172.21.15.93:0/1298279218 wait complete. 2023-12-14T18:24:10.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b97421700 1 Processor -- start 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b97421700 1 -- start start 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b97421700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b900feab0 0x7f8b9019c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b97421700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 0x7f8b9019cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b97421700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b90103dd0 0x7f8b901a3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b97421700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b901a41d0 con 0x7f8b90101410 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b97421700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b901a4310 con 0x7f8b90103dd0 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b97421700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b901a44a0 con 0x7f8b900feab0 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b949bc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 0x7f8b9019cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b949bc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 0x7f8b9019cb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36092/0 (socket says 172.21.15.93:36092) 2023-12-14T18:24:10.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b949bc700 1 -- 172.21.15.93:0/654542578 learned_addr learned my addr 172.21.15.93:0/654542578 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b951bd700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b900feab0 0x7f8b9019c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b949bc700 1 -- 172.21.15.93:0/654542578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b900feab0 msgr2=0x7f8b9019c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b949bc700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b900feab0 0x7f8b9019c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b949bc700 1 -- 172.21.15.93:0/654542578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b90103dd0 msgr2=0x7f8b901a3c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b949bc700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b90103dd0 0x7f8b901a3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.479+0000 7f8b949bc700 1 -- 172.21.15.93:0/654542578 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b800095f0 con 0x7f8b90101410 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.480+0000 7f8b949bc700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 0x7f8b9019cb90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8b80006010 tx=0x7f8b8000f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.480+0000 7f8b867fc700 1 -- 172.21.15.93:0/654542578 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b8001d070 con 0x7f8b90101410 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.480+0000 7f8b867fc700 1 -- 172.21.15.93:0/654542578 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b8000fbf0 con 0x7f8b90101410 2023-12-14T18:24:10.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.480+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b90112260 con 0x7f8b90101410 2023-12-14T18:24:10.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.480+0000 7f8b867fc700 1 -- 172.21.15.93:0/654542578 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b80017650 con 0x7f8b90101410 2023-12-14T18:24:10.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.480+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b901120d0 con 0x7f8b90101410 2023-12-14T18:24:10.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.481+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b900fc2b0 con 0x7f8b90101410 2023-12-14T18:24:10.487 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.482+0000 7f8b867fc700 1 -- 172.21.15.93:0/654542578 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8b800177b0 con 0x7f8b90101410 2023-12-14T18:24:10.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.482+0000 7f8b867fc700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b7c0a1150 0x7f8b7c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.483+0000 7f8b951bd700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b7c0a1150 0x7f8b7c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.483+0000 7f8b867fc700 1 -- 172.21.15.93:0/654542578 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8b800cd700 con 0x7f8b90101410 2023-12-14T18:24:10.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.483+0000 7f8b951bd700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b7c0a1150 0x7f8b7c0a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c005950 tx=0x7f8b8c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:10.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.487+0000 7f8b867fc700 1 -- 172.21.15.93:0/654542578 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8b80097380 con 0x7f8b90101410 2023-12-14T18:24:10.657 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi160=c;smithi160=f;smithi160=i;smithi160=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:10.657 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: Saving service mds.all spec with placement smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi160=c;smithi160=f;smithi160=i;smithi160=l;count:12 2023-12-14T18:24:10.657 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:10.657 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:10.657 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: Deploying daemon mds.l on smithi160 2023-12-14T18:24:10.658 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:10.659 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:10 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1227386039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:10.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.673+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8b90061700 con 0x7f8b90101410 2023-12-14T18:24:10.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.673+0000 7f8b867fc700 1 -- 172.21.15.93:0/654542578 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8b8009a9a0 con 0x7f8b90101410 2023-12-14T18:24:10.675 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:24:10.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b7c0a1150 msgr2=0x7f8b7c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b7c0a1150 0x7f8b7c0a3600 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c005950 tx=0x7f8b8c00a300 comp rx=0 tx=0).stop 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 msgr2=0x7f8b9019cb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 0x7f8b9019cb90 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8b80006010 tx=0x7f8b8000f700 comp rx=0 tx=0).stop 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 shutdown_connections 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b7c0a1150 0x7f8b7c0a3600 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b90103dd0 0x7f8b901a3c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b900feab0 0x7f8b9019c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 --2- 172.21.15.93:0/654542578 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b90101410 0x7f8b9019cb90 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 >> 172.21.15.93:0/654542578 conn(0x7f8b900fa740 msgr2=0x7f8b900ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 shutdown_connections 2023-12-14T18:24:10.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.675+0000 7f8b97421700 1 -- 172.21.15.93:0/654542578 wait complete. 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi160=c;smithi160=f;smithi160=i;smithi160=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: Saving service mds.all spec with placement smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi160=c;smithi160=f;smithi160=i;smithi160=l;count:12 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:24:10.689 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:24:10.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:24:10.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:10.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:10.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:10.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: Deploying daemon mds.l on smithi160 2023-12-14T18:24:10.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:10.690 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:10 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1227386039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:10.691 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:10.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 -- 172.21.15.93:0/2801145619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100cd0 msgr2=0x7fdaf8107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 --2- 172.21.15.93:0/2801145619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100cd0 0x7fdaf8107590 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fdaf000b0a0 tx=0x7fdaf000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 -- 172.21.15.93:0/2801145619 shutdown_connections 2023-12-14T18:24:10.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 --2- 172.21.15.93:0/2801145619 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100cd0 0x7fdaf8107590 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 --2- 172.21.15.93:0/2801145619 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf80ff150 0x7fdaf80ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 --2- 172.21.15.93:0/2801145619 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaf8100340 0x7fdaf8100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 -- 172.21.15.93:0/2801145619 >> 172.21.15.93:0/2801145619 conn(0x7fdaf80fa750 msgr2=0x7fdaf80fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 -- 172.21.15.93:0/2801145619 shutdown_connections 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 -- 172.21.15.93:0/2801145619 wait complete. 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 Processor -- start 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 -- start start 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaf80ff150 0x7fdaf8069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100340 0x7fdaf819ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf8100cd0 0x7fdaf819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf819dc00 con 0x7fdaf8100340 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf819dd70 con 0x7fdaf8100cd0 2023-12-14T18:24:10.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.813+0000 7fdaff40b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaf819df30 con 0x7fdaf80ff150 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafc9a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100340 0x7fdaf819ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafc9a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100340 0x7fdaf819ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36102/0 (socket says 172.21.15.93:36102) 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafc9a6700 1 -- 172.21.15.93:0/3587293131 learned_addr learned my addr 172.21.15.93:0/3587293131 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafd1a7700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaf80ff150 0x7fdaf8069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafd9a8700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf8100cd0 0x7fdaf819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafc9a6700 1 -- 172.21.15.93:0/3587293131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaf80ff150 msgr2=0x7fdaf8069b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafc9a6700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaf80ff150 0x7fdaf8069b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafc9a6700 1 -- 172.21.15.93:0/3587293131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf8100cd0 msgr2=0x7fdaf819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafc9a6700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf8100cd0 0x7fdaf819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafc9a6700 1 -- 172.21.15.93:0/3587293131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaf0009d00 con 0x7fdaf8100340 2023-12-14T18:24:10.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdafc9a6700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100340 0x7fdaf819ade0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400e9a0 tx=0x7fdaf400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:10.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdaee7fc700 1 -- 172.21.15.93:0/3587293131 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4009790 con 0x7fdaf8100340 2023-12-14T18:24:10.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdaee7fc700 1 -- 172.21.15.93:0/3587293131 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdaf4010430 con 0x7fdaf8100340 2023-12-14T18:24:10.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaf810aac0 con 0x7fdaf8100340 2023-12-14T18:24:10.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdaee7fc700 1 -- 172.21.15.93:0/3587293131 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaf4018680 con 0x7fdaf8100340 2023-12-14T18:24:10.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.814+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaf810a930 con 0x7fdaf8100340 2023-12-14T18:24:10.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.815+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaf81066a0 con 0x7fdaf8100340 2023-12-14T18:24:10.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.816+0000 7fdaee7fc700 1 -- 172.21.15.93:0/3587293131 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fdaf40105a0 con 0x7fdaf8100340 2023-12-14T18:24:10.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.817+0000 7fdaee7fc700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdae40a11d0 0x7fdae40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:10.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.817+0000 7fdaee7fc700 1 -- 172.21.15.93:0/3587293131 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fdaf4014070 con 0x7fdaf8100340 2023-12-14T18:24:10.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.817+0000 7fdafd1a7700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdae40a11d0 0x7fdae40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:10.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.817+0000 7fdafd1a7700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdae40a11d0 0x7fdae40a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdaf81001a0 tx=0x7fdae800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:10.823 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:10.820+0000 7fdaee7fc700 1 -- 172.21.15.93:0/3587293131 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdaf40963d0 con 0x7fdaf8100340 2023-12-14T18:24:10.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='client.24364 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi160=c;smithi160=f;smithi160=i;smithi160=l", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:24:10.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: Saving service mds.all spec with placement smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi160=c;smithi160=f;smithi160=i;smithi160=l;count:12 2023-12-14T18:24:10.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:10.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:10.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:10.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:10.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:24:10.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:10.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:24:10.844 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:24:10.844 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:24:10.844 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:10.844 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:10.844 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:10.844 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: Deploying daemon mds.l on smithi160 2023-12-14T18:24:10.844 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:10.844 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:10 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1227386039' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:11.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.053+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaf802cfe0 con 0x7fdaf8100340 2023-12-14T18:24:11.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.053+0000 7fdaee7fc700 1 -- 172.21.15.93:0/3587293131 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) v1 ==== 93+0+501 (secure 0 0 0) 0x7fdaf4026210 con 0x7fdaf8100340 2023-12-14T18:24:11.055 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:11.055 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdae40a11d0 msgr2=0x7fdae40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdae40a11d0 0x7fdae40a3680 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fdaf81001a0 tx=0x7fdae800b410 comp rx=0 tx=0).stop 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100340 msgr2=0x7fdaf819ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100340 0x7fdaf819ade0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400e9a0 tx=0x7fdaf400ecb0 comp rx=0 tx=0).stop 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 shutdown_connections 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fdae40a11d0 0x7fdae40a3680 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdaf8100cd0 0x7fdaf819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdaf80ff150 0x7fdaf8069b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 --2- 172.21.15.93:0/3587293131 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdaf8100340 0x7fdaf819ade0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:11.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 >> 172.21.15.93:0/3587293131 conn(0x7fdaf80fa750 msgr2=0x7fdaf81037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 shutdown_connections 2023-12-14T18:24:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:11.055+0000 7fdaff40b700 1 -- 172.21.15.93:0/3587293131 wait complete. 2023-12-14T18:24:11.059 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 1 2023-12-14T18:24:11.366 INFO:journalctl@ceph.mds.l.smithi160.stdout:Dec 14 18:24:11 smithi160 systemd[1]: Starting Ceph mds.l for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:11.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:11 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/654542578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:11.748 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:11 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3587293131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:11.748 INFO:journalctl@ceph.mds.l.smithi160.stdout:Dec 14 18:24:11 smithi160 ceph-mds[68642]: -- 172.21.15.160:0/2638765816 <== mon.1 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55be541b3520 con 0x55be54189000 2023-12-14T18:24:11.748 INFO:journalctl@ceph.mds.l.smithi160.stdout:Dec 14 18:24:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-l[68618]: starting mds.l at 2023-12-14T18:24:11.748 INFO:journalctl@ceph.mds.l.smithi160.stdout:Dec 14 18:24:11 smithi160 bash[68603]: 6b04e95430d378d5cdd4a0883ae70c833dec23124b48f682047cdcab287439b9 2023-12-14T18:24:11.748 INFO:journalctl@ceph.mds.l.smithi160.stdout:Dec 14 18:24:11 smithi160 systemd[1]: Started Ceph mds.l for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:11.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:11 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/654542578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:11 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3587293131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:11.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:11 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/654542578' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:11.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:11 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3587293131' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:12.741 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.160:6832/2184947052,v1:172.21.15.160:6833/2184947052] up:boot 2023-12-14T18:24:12.741 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: fsmap 1 up:standby 2023-12-14T18:24:12.741 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:24:12.741 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:12.742 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:12.742 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:12.742 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:12.742 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:12.742 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:12.742 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: Deploying daemon mds.k on smithi097 2023-12-14T18:24:12.742 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:12 smithi097 ceph-mon[43486]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:12.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.160:6832/2184947052,v1:172.21.15.160:6833/2184947052] up:boot 2023-12-14T18:24:12.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: fsmap 1 up:standby 2023-12-14T18:24:12.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:24:12.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:12.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:12.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:12.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:12.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:12.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:12.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: Deploying daemon mds.k on smithi097 2023-12-14T18:24:12.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:12 smithi093 ceph-mon[42070]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:12.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.160:6832/2184947052,v1:172.21.15.160:6833/2184947052] up:boot 2023-12-14T18:24:12.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: fsmap 1 up:standby 2023-12-14T18:24:12.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:24:12.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:12.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:12.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:12.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:12.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:12.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:12.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: Deploying daemon mds.k on smithi097 2023-12-14T18:24:12.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:12 smithi160 ceph-mon[43840]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:13.070 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:13.081 INFO:journalctl@ceph.mds.k.smithi097.stdout:Dec 14 18:24:12 smithi097 systemd[1]: Starting Ceph mds.k for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:13.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.212+0000 7f3be5861700 1 -- 172.21.15.93:0/1464333611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 msgr2=0x7f3be0100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:13.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.212+0000 7f3be5861700 1 --2- 172.21.15.93:0/1464333611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 0x7f3be0100bd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f3bd4009990 tx=0x7f3bd4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.212+0000 7f3be5861700 1 -- 172.21.15.93:0/1464333611 shutdown_connections 2023-12-14T18:24:13.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.212+0000 7f3be5861700 1 --2- 172.21.15.93:0/1464333611 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 0x7f3be0100bd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.212+0000 7f3be5861700 1 --2- 172.21.15.93:0/1464333611 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be00ffd30 0x7f3be01075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.212+0000 7f3be5861700 1 --2- 172.21.15.93:0/1464333611 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3be00ff380 0x7f3be00ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.212+0000 7f3be5861700 1 -- 172.21.15.93:0/1464333611 >> 172.21.15.93:0/1464333611 conn(0x7f3be00f9ea0 msgr2=0x7f3be00fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:13.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.212+0000 7f3be5861700 1 -- 172.21.15.93:0/1464333611 shutdown_connections 2023-12-14T18:24:13.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.212+0000 7f3be5861700 1 -- 172.21.15.93:0/1464333611 wait complete. 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3be5861700 1 Processor -- start 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3be5861700 1 -- start start 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3be5861700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be00ff380 0x7f3be0198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3be5861700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3be00ffd30 0x7f3be01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3be5861700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 0x7f3be019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3be5861700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019fd00 con 0x7f3be01007c0 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3be5861700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019fe40 con 0x7f3be00ff380 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3be5861700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3be019ffd0 con 0x7f3be00ffd30 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3bdf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 0x7f3be019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.213+0000 7f3bdeffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be00ff380 0x7f3be0198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bdf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 0x7f3be019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33588/0 (socket says 172.21.15.93:33588) 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bdf7fe700 1 -- 172.21.15.93:0/3498278583 learned_addr learned my addr 172.21.15.93:0/3498278583 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bde7fc700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3be00ffd30 0x7f3be01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:13.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bdf7fe700 1 -- 172.21.15.93:0/3498278583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3be00ffd30 msgr2=0x7f3be01986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bdf7fe700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3be00ffd30 0x7f3be01986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bdf7fe700 1 -- 172.21.15.93:0/3498278583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be00ff380 msgr2=0x7f3be0198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bdf7fe700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be00ff380 0x7f3be0198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bdf7fe700 1 -- 172.21.15.93:0/3498278583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3bd40095f0 con 0x7f3be01007c0 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bdeffd700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be00ff380 0x7f3be0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3bdf7fe700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 0x7f3be019f730 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800ef30 tx=0x7f3bd800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3be485f700 1 -- 172.21.15.93:0/3498278583 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd8018070 con 0x7f3be01007c0 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3be485f700 1 -- 172.21.15.93:0/3498278583 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3bd800f040 con 0x7f3be01007c0 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3be010aae0 con 0x7f3be01007c0 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3be485f700 1 -- 172.21.15.93:0/3498278583 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3bd800cdb0 con 0x7f3be01007c0 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.214+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3be010a950 con 0x7f3be01007c0 2023-12-14T18:24:13.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.215+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3be0067e40 con 0x7f3be01007c0 2023-12-14T18:24:13.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.216+0000 7f3be485f700 1 -- 172.21.15.93:0/3498278583 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f3bd8003a00 con 0x7f3be01007c0 2023-12-14T18:24:13.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.217+0000 7f3be485f700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3bc80a1150 0x7f3bc80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:13.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.217+0000 7f3bdeffd700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3bc80a1150 0x7f3bc80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:13.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.217+0000 7f3be485f700 1 -- 172.21.15.93:0/3498278583 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f3bd80cc9c0 con 0x7f3be01007c0 2023-12-14T18:24:13.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.218+0000 7f3bdeffd700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3bc80a1150 0x7f3bc80a3600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400b2e0 tx=0x7f3bd4005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:13.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.221+0000 7f3be485f700 1 -- 172.21.15.93:0/3498278583 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3bd8096640 con 0x7f3be01007c0 2023-12-14T18:24:13.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.470+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3be01a0e40 con 0x7f3be01007c0 2023-12-14T18:24:13.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.470+0000 7f3be485f700 1 -- 172.21.15.93:0/3498278583 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1260 (secure 0 0 0) 0x7f3bd8099c60 con 0x7f3be01007c0 2023-12-14T18:24:13.473 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:13.473 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-14T18:24:13.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3bc80a1150 msgr2=0x7f3bc80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:13.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3bc80a1150 0x7f3bc80a3600 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3bd400b2e0 tx=0x7f3bd4005fb0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 msgr2=0x7f3be019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:13.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 0x7f3be019f730 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3bd800ef30 tx=0x7f3bd800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 shutdown_connections 2023-12-14T18:24:13.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3bc80a1150 0x7f3bc80a3600 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3be01007c0 0x7f3be019f730 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3be00ff380 0x7f3be0198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 --2- 172.21.15.93:0/3498278583 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3be00ffd30 0x7f3be01986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:13.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 >> 172.21.15.93:0/3498278583 conn(0x7f3be00f9ea0 msgr2=0x7f3be00fe2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:13.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 shutdown_connections 2023-12-14T18:24:13.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:13.473+0000 7f3be5861700 1 -- 172.21.15.93:0/3498278583 wait complete. 2023-12-14T18:24:13.477 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 2 2023-12-14T18:24:13.593 INFO:journalctl@ceph.mds.k.smithi097.stdout:Dec 14 18:24:13 smithi097 ceph-mds[67560]: -- 172.21.15.97:0/2108210766 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a5bff07520 con 0x55a5bfedc800 2023-12-14T18:24:13.593 INFO:journalctl@ceph.mds.k.smithi097.stdout:Dec 14 18:24:13 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-k[67537]: starting mds.k at 2023-12-14T18:24:13.593 INFO:journalctl@ceph.mds.k.smithi097.stdout:Dec 14 18:24:13 smithi097 bash[67522]: 8b10cd3c51550f828bd4f6de278f9ad7ac3b84ee7b7415e37628fa38bcfe0aae 2023-12-14T18:24:13.593 INFO:journalctl@ceph.mds.k.smithi097.stdout:Dec 14 18:24:13 smithi097 systemd[1]: Started Ceph mds.k for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:14.473 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:14.473 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:14.473 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:14.473 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:14.473 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:14.473 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:14.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: Deploying daemon mds.j on smithi093 2023-12-14T18:24:14.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3498278583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:14.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.97:6832/1737809375,v1:172.21.15.97:6833/1737809375] up:boot 2023-12-14T18:24:14.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: fsmap 2 up:standby 2023-12-14T18:24:14.474 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:24:14.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: Deploying daemon mds.j on smithi093 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3498278583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.97:6832/1737809375,v1:172.21.15.97:6833/1737809375] up:boot 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: fsmap 2 up:standby 2023-12-14T18:24:14.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:14 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:24:14.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:14.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:14.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:14.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:14.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:14.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:14.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: Deploying daemon mds.j on smithi093 2023-12-14T18:24:14.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3498278583' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:14.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.97:6832/1737809375,v1:172.21.15.97:6833/1737809375] up:boot 2023-12-14T18:24:14.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: fsmap 2 up:standby 2023-12-14T18:24:14.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:14 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:24:14.771 INFO:journalctl@ceph.mds.j.smithi093.stdout:Dec 14 18:24:14 smithi093 systemd[1]: Starting Ceph mds.j for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:15.065 INFO:journalctl@ceph.mds.j.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-mds[75035]: -- 172.21.15.93:0/90864153 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x564aaf369520 con 0x564aaf33ec00 2023-12-14T18:24:15.065 INFO:journalctl@ceph.mds.j.smithi093.stdout:Dec 14 18:24:14 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-j[75012]: starting mds.j at 2023-12-14T18:24:15.065 INFO:journalctl@ceph.mds.j.smithi093.stdout:Dec 14 18:24:14 smithi093 bash[74997]: a5483b8f4310fdfa6b5689426a383dac73bbf41ab985f27ef5efda17420a65f6 2023-12-14T18:24:15.065 INFO:journalctl@ceph.mds.j.smithi093.stdout:Dec 14 18:24:14 smithi093 systemd[1]: Started Ceph mds.j for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:15.488 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:15.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:15 smithi093 ceph-mon[42070]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:15.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:15.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:15.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:15.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:15.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:15.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:15 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:15.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:15 smithi097 ceph-mon[43486]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:15.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:15 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:15.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.622+0000 7f2c71470700 1 -- 172.21.15.93:0/4241863339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 msgr2=0x7f2c6c0fe330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:15.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.622+0000 7f2c71470700 1 --2- 172.21.15.93:0/4241863339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 0x7f2c6c0fe330 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f2c60009990 tx=0x7f2c60009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.622+0000 7f2c71470700 1 -- 172.21.15.93:0/4241863339 shutdown_connections 2023-12-14T18:24:15.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.622+0000 7f2c71470700 1 --2- 172.21.15.93:0/4241863339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6c0fe950 0x7f2c6c0fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.622+0000 7f2c71470700 1 --2- 172.21.15.93:0/4241863339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c6c1018c0 0x7f2c6c101cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.622+0000 7f2c71470700 1 --2- 172.21.15.93:0/4241863339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 0x7f2c6c0fe330 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.622+0000 7f2c71470700 1 -- 172.21.15.93:0/4241863339 >> 172.21.15.93:0/4241863339 conn(0x7f2c6c0f9ee0 msgr2=0x7f2c6c0fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:15.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.622+0000 7f2c71470700 1 -- 172.21.15.93:0/4241863339 shutdown_connections 2023-12-14T18:24:15.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.622+0000 7f2c71470700 1 -- 172.21.15.93:0/4241863339 wait complete. 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c71470700 1 Processor -- start 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c71470700 1 -- start start 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c71470700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6c0fe950 0x7f2c6c195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c71470700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c6c1018c0 0x7f2c6c196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c71470700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 0x7f2c6c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c71470700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c19db60 con 0x7f2c6c102210 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c71470700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c19dca0 con 0x7f2c6c0fe950 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c71470700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c6c19de30 con 0x7f2c6c1018c0 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 0x7f2c6c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c6a7fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c6c1018c0 0x7f2c6c196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:15.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c6b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 0x7f2c6c19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33620/0 (socket says 172.21.15.93:33620) 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c6b7fe700 1 -- 172.21.15.93:0/4187535497 learned_addr learned my addr 172.21.15.93:0/4187535497 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c6affd700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6c0fe950 0x7f2c6c195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.623+0000 7f2c6b7fe700 1 -- 172.21.15.93:0/4187535497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c6c1018c0 msgr2=0x7f2c6c196520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c6b7fe700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c6c1018c0 0x7f2c6c196520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c6b7fe700 1 -- 172.21.15.93:0/4187535497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6c0fe950 msgr2=0x7f2c6c195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c6b7fe700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6c0fe950 0x7f2c6c195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c6b7fe700 1 -- 172.21.15.93:0/4187535497 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c600095f0 con 0x7f2c6c102210 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c6b7fe700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 0x7f2c6c19d590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c64007ee0 tx=0x7f2c6400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c53fff700 1 -- 172.21.15.93:0/4187535497 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64017070 con 0x7f2c6c102210 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c6c10bc10 con 0x7f2c6c102210 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c53fff700 1 -- 172.21.15.93:0/4187535497 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c6400f040 con 0x7f2c6c102210 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c53fff700 1 -- 172.21.15.93:0/4187535497 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c64005740 con 0x7f2c6c102210 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.624+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c6c10ba80 con 0x7f2c6c102210 2023-12-14T18:24:15.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.625+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c6c107030 con 0x7f2c6c102210 2023-12-14T18:24:15.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.626+0000 7f2c53fff700 1 -- 172.21.15.93:0/4187535497 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c6400dc60 con 0x7f2c6c102210 2023-12-14T18:24:15.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.626+0000 7f2c53fff700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c540a1150 0x7f2c540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:15.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.627+0000 7f2c53fff700 1 -- 172.21.15.93:0/4187535497 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f2c640cbcb0 con 0x7f2c6c102210 2023-12-14T18:24:15.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.627+0000 7f2c6affd700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c540a1150 0x7f2c540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:15.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.627+0000 7f2c6affd700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c540a1150 0x7f2c540a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c005950 tx=0x7f2c5c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:15.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.631+0000 7f2c53fff700 1 -- 172.21.15.93:0/4187535497 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2c640958a0 con 0x7f2c6c102210 2023-12-14T18:24:15.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:15 smithi160 ceph-mon[43840]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:15.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:15.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:15.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:15.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:15.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:15.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:15 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:15.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.876+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c6c067e40 con 0x7f2c6c102210 2023-12-14T18:24:15.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.877+0000 7f2c53fff700 1 -- 172.21.15.93:0/4187535497 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) v1 ==== 93+0+2768 (secure 0 0 0) 0x7f2c64095020 con 0x7f2c6c102210 2023-12-14T18:24:15.879 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:15.879 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c540a1150 msgr2=0x7f2c540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c540a1150 0x7f2c540a3600 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2c5c005950 tx=0x7f2c5c0058e0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 msgr2=0x7f2c6c19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 0x7f2c6c19d590 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f2c64007ee0 tx=0x7f2c6400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 shutdown_connections 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c540a1150 0x7f2c540a3600 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2c6c102210 0x7f2c6c19d590 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2c6c0fe950 0x7f2c6c195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 --2- 172.21.15.93:0/4187535497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2c6c1018c0 0x7f2c6c196520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 >> 172.21.15.93:0/4187535497 conn(0x7f2c6c0f9ee0 msgr2=0x7f2c6c0fbbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:15.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 shutdown_connections 2023-12-14T18:24:15.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:15.879+0000 7f2c71470700 1 -- 172.21.15.93:0/4187535497 wait complete. 2023-12-14T18:24:15.882 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 4 2023-12-14T18:24:16.565 INFO:journalctl@ceph.mds.d.smithi093.stdout:Dec 14 18:24:16 smithi093 systemd[1]: Starting Ceph mds.d for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:16.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:16 smithi093 ceph-mon[42070]: Deploying daemon mds.d on smithi093 2023-12-14T18:24:16.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:16 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.93:6834/53835883,v1:172.21.15.93:6835/53835883] up:boot 2023-12-14T18:24:16.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:16 smithi093 ceph-mon[42070]: fsmap 3 up:standby 2023-12-14T18:24:16.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:16 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:24:16.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:16 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4187535497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:16.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:16 smithi097 ceph-mon[43486]: Deploying daemon mds.d on smithi093 2023-12-14T18:24:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:16 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.93:6834/53835883,v1:172.21.15.93:6835/53835883] up:boot 2023-12-14T18:24:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:16 smithi097 ceph-mon[43486]: fsmap 3 up:standby 2023-12-14T18:24:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:16 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:24:16.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:16 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4187535497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:16.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:16 smithi160 ceph-mon[43840]: Deploying daemon mds.d on smithi093 2023-12-14T18:24:16.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:16 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.93:6834/53835883,v1:172.21.15.93:6835/53835883] up:boot 2023-12-14T18:24:16.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:16 smithi160 ceph-mon[43840]: fsmap 3 up:standby 2023-12-14T18:24:16.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:16 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:24:16.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:16 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4187535497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:17.064 INFO:journalctl@ceph.mds.d.smithi093.stdout:Dec 14 18:24:16 smithi093 ceph-mds[75423]: -- 172.21.15.93:0/1072889717 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x555980063520 con 0x555980038800 2023-12-14T18:24:17.065 INFO:journalctl@ceph.mds.d.smithi093.stdout:Dec 14 18:24:16 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-d[75399]: starting mds.d at 2023-12-14T18:24:17.065 INFO:journalctl@ceph.mds.d.smithi093.stdout:Dec 14 18:24:16 smithi093 bash[75385]: e1e62891400cfa9bf1f6aa2f33c3f5598235519c7ed287c7a5066ea02b9eb042 2023-12-14T18:24:17.065 INFO:journalctl@ceph.mds.d.smithi093.stdout:Dec 14 18:24:16 smithi093 systemd[1]: Started Ceph mds.d for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:17.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:17 smithi093 ceph-mon[42070]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:17.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:17.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:17.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:17.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:17.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:17.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:17 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:17.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:17 smithi097 ceph-mon[43486]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:17.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:17 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:17.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:17 smithi160 ceph-mon[43840]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:17.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:17.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:17.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:17.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:17.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:17.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:17 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:17.896 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.040+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/641545805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100ca0 msgr2=0x7f8fd8107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.040+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/641545805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100ca0 0x7f8fd8107560 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f8fd000b0a0 tx=0x7f8fd000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.040+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/641545805 shutdown_connections 2023-12-14T18:24:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.040+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/641545805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100ca0 0x7f8fd8107560 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.040+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/641545805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd80ff120 0x7f8fd80ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.040+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/641545805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd8100310 0x7f8fd8100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.040+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/641545805 >> 172.21.15.93:0/641545805 conn(0x7f8fd80fa700 msgr2=0x7f8fd80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.040+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/641545805 shutdown_connections 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.040+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/641545805 wait complete. 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdeb4d700 1 Processor -- start 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdeb4d700 1 -- start start 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdeb4d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd80ff120 0x7f8fd80699e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdeb4d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100310 0x7f8fd819ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdeb4d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd8100ca0 0x7f8fd819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdeb4d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd819dbc0 con 0x7f8fd8100310 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdeb4d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd819dd30 con 0x7f8fd80ff120 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdeb4d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fd819def0 con 0x7f8fd8100ca0 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fd7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100310 0x7f8fd819ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fd7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100310 0x7f8fd819ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33654/0 (socket says 172.21.15.93:33654) 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fd7fff700 1 -- 172.21.15.93:0/1880216100 learned_addr learned my addr 172.21.15.93:0/1880216100 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdc8e9700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd80ff120 0x7f8fd80699e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:18.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdd0ea700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd8100ca0 0x7f8fd819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fd7fff700 1 -- 172.21.15.93:0/1880216100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd8100ca0 msgr2=0x7f8fd819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fd7fff700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd8100ca0 0x7f8fd819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fd7fff700 1 -- 172.21.15.93:0/1880216100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd80ff120 msgr2=0x7f8fd80699e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fd7fff700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd80ff120 0x7f8fd80699e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fd7fff700 1 -- 172.21.15.93:0/1880216100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fd0009d00 con 0x7f8fd8100310 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fdd0ea700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd8100ca0 0x7f8fd819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.041+0000 7f8fd7fff700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100310 0x7f8fd819ada0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d980 tx=0x7f8fc400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.042+0000 7f8fd5ffb700 1 -- 172.21.15.93:0/1880216100 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4004220 con 0x7f8fd8100310 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.042+0000 7f8fd5ffb700 1 -- 172.21.15.93:0/1880216100 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8fc4007640 con 0x7f8fd8100310 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.042+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fd810aa90 con 0x7f8fd8100310 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.042+0000 7f8fd5ffb700 1 -- 172.21.15.93:0/1880216100 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4005610 con 0x7f8fd8100310 2023-12-14T18:24:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.042+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fd810a900 con 0x7f8fd8100310 2023-12-14T18:24:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.043+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fd8106640 con 0x7f8fd8100310 2023-12-14T18:24:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.044+0000 7f8fd5ffb700 1 -- 172.21.15.93:0/1880216100 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8fc4004750 con 0x7f8fd8100310 2023-12-14T18:24:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.044+0000 7f8fd5ffb700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8fc80a11d0 0x7f8fc80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.044+0000 7f8fdc8e9700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8fc80a11d0 0x7f8fc80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.044+0000 7f8fd5ffb700 1 -- 172.21.15.93:0/1880216100 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f8fc40cc220 con 0x7f8fd8100310 2023-12-14T18:24:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.045+0000 7f8fdc8e9700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8fc80a11d0 0x7f8fc80a3680 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8100170 tx=0x7f8fcc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:18.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.049+0000 7f8fd5ffb700 1 -- 172.21.15.93:0/1880216100 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8fc4095e10 con 0x7f8fd8100310 2023-12-14T18:24:18.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.279+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fd8061b20 con 0x7f8fd8100310 2023-12-14T18:24:18.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.279+0000 7f8fd5ffb700 1 -- 172.21.15.93:0/1880216100 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) v1 ==== 93+0+3525 (secure 0 0 0) 0x7f8fc4016de0 con 0x7f8fd8100310 2023-12-14T18:24:18.281 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:18.281 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-12-14T18:24:18.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8fc80a11d0 msgr2=0x7f8fc80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:18.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8fc80a11d0 0x7f8fc80a3680 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f8fd8100170 tx=0x7f8fcc00b410 comp rx=0 tx=0).stop 2023-12-14T18:24:18.283 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100310 msgr2=0x7f8fd819ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100310 0x7f8fd819ada0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f8fc400d980 tx=0x7f8fc400dc90 comp rx=0 tx=0).stop 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 shutdown_connections 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8fc80a11d0 0x7f8fc80a3680 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8fd8100ca0 0x7f8fd819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8fd80ff120 0x7f8fd80699e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 --2- 172.21.15.93:0/1880216100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8fd8100310 0x7f8fd819ada0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.281+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 >> 172.21.15.93:0/1880216100 conn(0x7f8fd80fa700 msgr2=0x7f8fd8103780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.282+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 shutdown_connections 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:18.282+0000 7f8fdeb4d700 1 -- 172.21.15.93:0/1880216100 wait complete. 2023-12-14T18:24:18.284 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 5 2023-12-14T18:24:18.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:18 smithi097 ceph-mon[43486]: Deploying daemon mds.e on smithi097 2023-12-14T18:24:18.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:18 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.93:6836/1107020110,v1:172.21.15.93:6837/1107020110] up:boot 2023-12-14T18:24:18.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:18 smithi097 ceph-mon[43486]: fsmap 4 up:standby 2023-12-14T18:24:18.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:18 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:24:18.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:18 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1880216100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:18.343 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:24:18 smithi097 systemd[1]: Starting Ceph mds.e for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:18.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:18 smithi093 ceph-mon[42070]: Deploying daemon mds.e on smithi097 2023-12-14T18:24:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:18 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.93:6836/1107020110,v1:172.21.15.93:6837/1107020110] up:boot 2023-12-14T18:24:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:18 smithi093 ceph-mon[42070]: fsmap 4 up:standby 2023-12-14T18:24:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:18 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:24:18.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:18 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1880216100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:18.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:18 smithi160 ceph-mon[43840]: Deploying daemon mds.e on smithi097 2023-12-14T18:24:18.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:18 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.93:6836/1107020110,v1:172.21.15.93:6837/1107020110] up:boot 2023-12-14T18:24:18.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:18 smithi160 ceph-mon[43840]: fsmap 4 up:standby 2023-12-14T18:24:18.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:18 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:24:18.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:18 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1880216100' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:18.842 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:24:18 smithi097 ceph-mds[67902]: -- 172.21.15.97:0/2656226533 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56309db35520 con 0x56309db0a800 2023-12-14T18:24:18.842 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:24:18 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-e[67879]: starting mds.e at 2023-12-14T18:24:18.842 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:24:18 smithi097 bash[67864]: 611c47ccc61f4197a652b5058352504de4e24a54fc6018bd6aac11de98fdeabe 2023-12-14T18:24:18.842 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:24:18 smithi097 systemd[1]: Started Ceph mds.e for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:19 smithi160 ceph-mon[43840]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:19.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:19.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:19.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:19 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:19.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:19 smithi093 ceph-mon[42070]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:19.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:19 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:19.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:19 smithi097 ceph-mon[43486]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:19.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:19.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:19 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:20.246 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:24:20 smithi160 systemd[1]: Starting Ceph mds.f for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:20.297 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.433+0000 7f7b6a2db700 1 -- 172.21.15.93:0/1710039685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100300 msgr2=0x7f7b64100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.433+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/1710039685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100300 0x7f7b64100750 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f7b58009990 tx=0x7f7b58009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.433+0000 7f7b6a2db700 1 -- 172.21.15.93:0/1710039685 shutdown_connections 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.433+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/1710039685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b64100c90 0x7f7b64107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.433+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/1710039685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b640ff110 0x7f7b640ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.433+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/1710039685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100300 0x7f7b64100750 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.433+0000 7f7b6a2db700 1 -- 172.21.15.93:0/1710039685 >> 172.21.15.93:0/1710039685 conn(0x7f7b640fa6f0 msgr2=0x7f7b640fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.433+0000 7f7b6a2db700 1 -- 172.21.15.93:0/1710039685 shutdown_connections 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.433+0000 7f7b6a2db700 1 -- 172.21.15.93:0/1710039685 wait complete. 2023-12-14T18:24:20.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b6a2db700 1 Processor -- start 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b6a2db700 1 -- start start 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b6a2db700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b640ff110 0x7f7b64195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b6a2db700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b64100300 0x7f7b64196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b6a2db700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100c90 0x7f7b6419d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b6a2db700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6419db70 con 0x7f7b64100c90 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b6a2db700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6419dcb0 con 0x7f7b640ff110 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b6a2db700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b6419de40 con 0x7f7b64100300 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b68878700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100c90 0x7f7b6419d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b68878700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100c90 0x7f7b6419d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33670/0 (socket says 172.21.15.93:33670) 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b68878700 1 -- 172.21.15.93:0/476568686 learned_addr learned my addr 172.21.15.93:0/476568686 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b63fff700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b640ff110 0x7f7b64195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b637fe700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b64100300 0x7f7b64196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b68878700 1 -- 172.21.15.93:0/476568686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b64100300 msgr2=0x7f7b64196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b68878700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b64100300 0x7f7b64196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b68878700 1 -- 172.21.15.93:0/476568686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b640ff110 msgr2=0x7f7b64195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:20.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b68878700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b640ff110 0x7f7b64195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.434+0000 7f7b68878700 1 -- 172.21.15.93:0/476568686 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b580095f0 con 0x7f7b64100c90 2023-12-14T18:24:20.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.435+0000 7f7b68878700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100c90 0x7f7b6419d5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c007ee0 tx=0x7f7b5c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:20.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.435+0000 7f7b617fa700 1 -- 172.21.15.93:0/476568686 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c017070 con 0x7f7b64100c90 2023-12-14T18:24:20.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.435+0000 7f7b617fa700 1 -- 172.21.15.93:0/476568686 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7b5c00f040 con 0x7f7b64100c90 2023-12-14T18:24:20.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.435+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7b6410aac0 con 0x7f7b64100c90 2023-12-14T18:24:20.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.435+0000 7f7b617fa700 1 -- 172.21.15.93:0/476568686 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b5c005580 con 0x7f7b64100c90 2023-12-14T18:24:20.438 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.435+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b6410a930 con 0x7f7b64100c90 2023-12-14T18:24:20.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.436+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b640679f0 con 0x7f7b64100c90 2023-12-14T18:24:20.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.437+0000 7f7b617fa700 1 -- 172.21.15.93:0/476568686 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7b5c00db20 con 0x7f7b64100c90 2023-12-14T18:24:20.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.437+0000 7f7b617fa700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:20.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.438+0000 7f7b63fff700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:20.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.438+0000 7f7b617fa700 1 -- 172.21.15.93:0/476568686 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f7b5c0cc250 con 0x7f7b64100c90 2023-12-14T18:24:20.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.438+0000 7f7b63fff700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7b64100160 tx=0x7f7b5400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:20.441 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.441+0000 7f7b617fa700 1 -- 172.21.15.93:0/476568686 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7b5c095e40 con 0x7f7b64100c90 2023-12-14T18:24:20.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:20 smithi093 ceph-mon[42070]: Deploying daemon mds.f on smithi160 2023-12-14T18:24:20.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:20 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] up:boot 2023-12-14T18:24:20.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:20 smithi093 ceph-mon[42070]: fsmap 5 up:standby 2023-12-14T18:24:20.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:20 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:24:20.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:20 smithi097 ceph-mon[43486]: Deploying daemon mds.f on smithi160 2023-12-14T18:24:20.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:20 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] up:boot 2023-12-14T18:24:20.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:20 smithi097 ceph-mon[43486]: fsmap 5 up:standby 2023-12-14T18:24:20.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:20 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:24:20.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.684+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7b6419ecf0 con 0x7f7b64100c90 2023-12-14T18:24:20.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.685+0000 7f7b617fa700 1 -- 172.21.15.93:0/476568686 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) v1 ==== 93+0+4282 (secure 0 0 0) 0x7f7b5c099460 con 0x7f7b64100c90 2023-12-14T18:24:20.686 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:20.686 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6}],"filesystems":[]} 2023-12-14T18:24:20.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7b4c0a1150 msgr2=0x7f7b4c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:20.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f7b64100160 tx=0x7f7b5400a380 comp rx=0 tx=0).stop 2023-12-14T18:24:20.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100c90 msgr2=0x7f7b6419d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:20.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100c90 0x7f7b6419d5a0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c007ee0 tx=0x7f7b5c00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 shutdown_connections 2023-12-14T18:24:20.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7b4c0a1150 0x7f7b4c0a3600 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7b64100c90 0x7f7b6419d5a0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7b640ff110 0x7f7b64195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 --2- 172.21.15.93:0/476568686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7b64100300 0x7f7b64196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:20.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 >> 172.21.15.93:0/476568686 conn(0x7f7b640fa6f0 msgr2=0x7f7b64067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:20.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 shutdown_connections 2023-12-14T18:24:20.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:20.687+0000 7f7b6a2db700 1 -- 172.21.15.93:0/476568686 wait complete. 2023-12-14T18:24:20.689 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 6 2023-12-14T18:24:20.746 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:24:20 smithi160 ceph-mds[68985]: -- 172.21.15.160:0/1319432698 <== mon.1 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a7c0d35520 con 0x55a7c0d0a800 2023-12-14T18:24:20.746 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:24:20 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-f[68962]: starting mds.f at 2023-12-14T18:24:20.746 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:24:20 smithi160 bash[68948]: 345a5a56e3763f52c2821d6016ef3e6c340b150f8bb26a4d9b047f1d89312348 2023-12-14T18:24:20.746 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:24:20 smithi160 systemd[1]: Started Ceph mds.f for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:20.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:20 smithi160 ceph-mon[43840]: Deploying daemon mds.f on smithi160 2023-12-14T18:24:20.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:20 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] up:boot 2023-12-14T18:24:20.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:20 smithi160 ceph-mon[43840]: fsmap 5 up:standby 2023-12-14T18:24:20.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:20 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:24:21.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:21 smithi093 ceph-mon[42070]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:21.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:21.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:21.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:21.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:21.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:21.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:21 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:21.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:21 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/476568686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:21.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:21 smithi097 ceph-mon[43486]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:21.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:21.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:21.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:21.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:21.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:21.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:21 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:21.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:21 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/476568686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:21.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:21 smithi160 ceph-mon[43840]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:21.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:21.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:21.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:21.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:21.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:21.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:21 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:21.695 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:21 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/476568686' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:21.996 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:24:21 smithi160 systemd[1]: Starting Ceph mds.c for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:22.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:22 smithi160 ceph-mon[43840]: Deploying daemon mds.c on smithi160 2023-12-14T18:24:22.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:22 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.160:6834/2285100381,v1:172.21.15.160:6835/2285100381] up:boot 2023-12-14T18:24:22.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:22 smithi160 ceph-mon[43840]: fsmap 6 up:standby 2023-12-14T18:24:22.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:22 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:24:22.497 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:24:22 smithi160 ceph-mds[69325]: -- 172.21.15.160:0/315092684 <== mon.1 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5571ef421520 con 0x5571ef3f6800 2023-12-14T18:24:22.497 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:24:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-c[69302]: starting mds.c at 2023-12-14T18:24:22.497 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:24:22 smithi160 bash[69287]: 6168261b32e6338fcfbde7bbea43b46c3e63a89904f7fe10f43b3f3508b839d6 2023-12-14T18:24:22.497 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:24:22 smithi160 systemd[1]: Started Ceph mds.c for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:22.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:22 smithi097 ceph-mon[43486]: Deploying daemon mds.c on smithi160 2023-12-14T18:24:22.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:22 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.160:6834/2285100381,v1:172.21.15.160:6835/2285100381] up:boot 2023-12-14T18:24:22.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:22 smithi097 ceph-mon[43486]: fsmap 6 up:standby 2023-12-14T18:24:22.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:22 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:24:22.701 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:22.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:22 smithi093 ceph-mon[42070]: Deploying daemon mds.c on smithi160 2023-12-14T18:24:22.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:22 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.160:6834/2285100381,v1:172.21.15.160:6835/2285100381] up:boot 2023-12-14T18:24:22.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:22 smithi093 ceph-mon[42070]: fsmap 6 up:standby 2023-12-14T18:24:22.730 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:22 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:24:22.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.833+0000 7f832a2c0700 1 -- 172.21.15.93:0/1055699676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324103690 msgr2=0x7f8324105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.833+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1055699676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324103690 0x7f8324105a70 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f8318009990 tx=0x7f8318009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.833+0000 7f832a2c0700 1 -- 172.21.15.93:0/1055699676 shutdown_connections 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.833+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1055699676 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8324105fb0 0x7f8324108390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.833+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1055699676 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8324100d70 0x7f8324103150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.833+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1055699676 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324103690 0x7f8324105a70 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.833+0000 7f832a2c0700 1 -- 172.21.15.93:0/1055699676 >> 172.21.15.93:0/1055699676 conn(0x7f83240fa750 msgr2=0x7f83240fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 -- 172.21.15.93:0/1055699676 shutdown_connections 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 -- 172.21.15.93:0/1055699676 wait complete. 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 Processor -- start 2023-12-14T18:24:22.835 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 -- start start 2023-12-14T18:24:22.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8324100d70 0x7f8324196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:22.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8324103690 0x7f8324196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:22.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324105fb0 0x7f832419d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832419dbd0 con 0x7f8324105fb0 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832419dd10 con 0x7f8324103690 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.834+0000 7f832a2c0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f832419dea0 con 0x7f8324100d70 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832885d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324105fb0 0x7f832419d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832885d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324105fb0 0x7f832419d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35642/0 (socket says 172.21.15.93:35642) 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832885d700 1 -- 172.21.15.93:0/1150911516 learned_addr learned my addr 172.21.15.93:0/1150911516 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f83237fe700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8324103690 0x7f8324196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f8323fff700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8324100d70 0x7f8324196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832885d700 1 -- 172.21.15.93:0/1150911516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8324100d70 msgr2=0x7f8324196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832885d700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8324100d70 0x7f8324196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832885d700 1 -- 172.21.15.93:0/1150911516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8324103690 msgr2=0x7f8324196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832885d700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8324103690 0x7f8324196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832885d700 1 -- 172.21.15.93:0/1150911516 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83180095f0 con 0x7f8324105fb0 2023-12-14T18:24:22.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f8323fff700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8324100d70 0x7f8324196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:22.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832885d700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324105fb0 0x7f832419d600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f831c00bf60 tx=0x7f831c00bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:22.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f83217fa700 1 -- 172.21.15.93:0/1150911516 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c010070 con 0x7f8324105fb0 2023-12-14T18:24:22.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f83217fa700 1 -- 172.21.15.93:0/1150911516 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f831c008a90 con 0x7f8324105fb0 2023-12-14T18:24:22.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f832410bc40 con 0x7f8324105fb0 2023-12-14T18:24:22.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f83217fa700 1 -- 172.21.15.93:0/1150911516 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f831c005610 con 0x7f8324105fb0 2023-12-14T18:24:22.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.835+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f832410bab0 con 0x7f8324105fb0 2023-12-14T18:24:22.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.836+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83240fc2c0 con 0x7f8324105fb0 2023-12-14T18:24:22.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.838+0000 7f83217fa700 1 -- 172.21.15.93:0/1150911516 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f831c0057b0 con 0x7f8324105fb0 2023-12-14T18:24:22.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.839+0000 7f83217fa700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f830c0a1150 0x7f830c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:22.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.839+0000 7f8323fff700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f830c0a1150 0x7f830c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:22.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.839+0000 7f83217fa700 1 -- 172.21.15.93:0/1150911516 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f831c0cbef0 con 0x7f8324105fb0 2023-12-14T18:24:22.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.839+0000 7f8323fff700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f830c0a1150 0x7f830c0a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8314005950 tx=0x7f8314005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:22.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:22.842+0000 7f83217fa700 1 -- 172.21.15.93:0/1150911516 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f831c095a40 con 0x7f8324105fb0 2023-12-14T18:24:23.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.074+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f83240619b0 con 0x7f8324105fb0 2023-12-14T18:24:23.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.075+0000 7f83217fa700 1 -- 172.21.15.93:0/1150911516 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+5799 (secure 0 0 0) 0x7f831c099060 con 0x7f8324105fb0 2023-12-14T18:24:23.076 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:23.076 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8}],"filesystems":[]} 2023-12-14T18:24:23.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f830c0a1150 msgr2=0x7f830c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:23.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f830c0a1150 0x7f830c0a3600 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f8314005950 tx=0x7f8314005f50 comp rx=0 tx=0).stop 2023-12-14T18:24:23.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324105fb0 msgr2=0x7f832419d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:23.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324105fb0 0x7f832419d600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f831c00bf60 tx=0x7f831c00bf90 comp rx=0 tx=0).stop 2023-12-14T18:24:23.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 shutdown_connections 2023-12-14T18:24:23.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f830c0a1150 0x7f830c0a3600 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:23.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8324105fb0 0x7f832419d600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:23.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8324100d70 0x7f8324196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:23.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 --2- 172.21.15.93:0/1150911516 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8324103690 0x7f8324196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:23.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 >> 172.21.15.93:0/1150911516 conn(0x7f83240fa750 msgr2=0x7f8324101a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:23.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 shutdown_connections 2023-12-14T18:24:23.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:23.077+0000 7f832a2c0700 1 -- 172.21.15.93:0/1150911516 wait complete. 2023-12-14T18:24:23.079 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 8 2023-12-14T18:24:23.507 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:23.507 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.160:6836/233011976,v1:172.21.15.160:6837/233011976] up:boot 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: fsmap 7 up:standby 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: Deploying daemon mds.h on smithi097 2023-12-14T18:24:23.508 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:23 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1150911516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:23.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:23.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:23.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.160:6836/233011976,v1:172.21.15.160:6837/233011976] up:boot 2023-12-14T18:24:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: fsmap 7 up:standby 2023-12-14T18:24:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:24:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: Deploying daemon mds.h on smithi097 2023-12-14T18:24:23.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:23 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1150911516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.160:6836/233011976,v1:172.21.15.160:6837/233011976] up:boot 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: fsmap 7 up:standby 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: Deploying daemon mds.h on smithi097 2023-12-14T18:24:23.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:23 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1150911516' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:23.842 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:24:23 smithi097 systemd[1]: Starting Ceph mds.h for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:24.342 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:24:24 smithi097 ceph-mds[68243]: -- 172.21.15.97:0/2100392279 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561901373520 con 0x561901348800 2023-12-14T18:24:24.343 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:24:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-h[68220]: starting mds.h at 2023-12-14T18:24:24.343 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:24:24 smithi097 bash[68205]: 2a637fbc3368116c1c30efe79cbc0a2453d5a65fcc5428f5187602158e04699f 2023-12-14T18:24:24.343 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:24:24 smithi097 systemd[1]: Started Ceph mds.h for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:25.090 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:25.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 -- 172.21.15.93:0/2658020819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0ffb80 msgr2=0x7fa27c0fffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:25.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 --2- 172.21.15.93:0/2658020819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0ffb80 0x7fa27c0fffd0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa270009990 tx=0x7fa270009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 -- 172.21.15.93:0/2658020819 shutdown_connections 2023-12-14T18:24:25.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 --2- 172.21.15.93:0/2658020819 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c100510 0x7fa27c106dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 --2- 172.21.15.93:0/2658020819 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27c0fe990 0x7fa27c0feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 --2- 172.21.15.93:0/2658020819 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0ffb80 0x7fa27c0fffd0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 -- 172.21.15.93:0/2658020819 >> 172.21.15.93:0/2658020819 conn(0x7fa27c0f9fb0 msgr2=0x7fa27c0fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:25.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 -- 172.21.15.93:0/2658020819 shutdown_connections 2023-12-14T18:24:25.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 -- 172.21.15.93:0/2658020819 wait complete. 2023-12-14T18:24:25.228 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.226+0000 7fa281528700 1 Processor -- start 2023-12-14T18:24:25.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa281528700 1 -- start start 2023-12-14T18:24:25.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa281528700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0fe990 0x7fa27c10a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:25.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa281528700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27c0ffb80 0x7fa27c108720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:25.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa281528700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c100510 0x7fa27c108c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:25.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa281528700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c109400 con 0x7fa27c0fe990 2023-12-14T18:24:25.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa281528700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c109570 con 0x7fa27c0ffb80 2023-12-14T18:24:25.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa281528700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27c1a0100 con 0x7fa27c100510 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0fe990 0x7fa27c10a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0fe990 0x7fa27c10a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35662/0 (socket says 172.21.15.93:35662) 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27affd700 1 -- 172.21.15.93:0/2308083500 learned_addr learned my addr 172.21.15.93:0/2308083500 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27b7fe700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c100510 0x7fa27c108c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27a7fc700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27c0ffb80 0x7fa27c108720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27affd700 1 -- 172.21.15.93:0/2308083500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c100510 msgr2=0x7fa27c108c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27affd700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c100510 0x7fa27c108c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27affd700 1 -- 172.21.15.93:0/2308083500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27c0ffb80 msgr2=0x7fa27c108720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27affd700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27c0ffb80 0x7fa27c108720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27affd700 1 -- 172.21.15.93:0/2308083500 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2700095f0 con 0x7fa27c0fe990 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27a7fc700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27c0ffb80 0x7fa27c108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27b7fe700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c100510 0x7fa27c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa27affd700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0fe990 0x7fa27c10a0e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00d980 tx=0x7fa26c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:25.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa263fff700 1 -- 172.21.15.93:0/2308083500 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c00de60 con 0x7fa27c0fe990 2023-12-14T18:24:25.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa263fff700 1 -- 172.21.15.93:0/2308083500 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa26c0044e0 con 0x7fa27c0fe990 2023-12-14T18:24:25.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27c10bd90 con 0x7fa27c0fe990 2023-12-14T18:24:25.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa263fff700 1 -- 172.21.15.93:0/2308083500 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c003be0 con 0x7fa27c0fe990 2023-12-14T18:24:25.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.227+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27c10bc00 con 0x7fa27c0fe990 2023-12-14T18:24:25.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.228+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa27c105f50 con 0x7fa27c0fe990 2023-12-14T18:24:25.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.230+0000 7fa263fff700 1 -- 172.21.15.93:0/2308083500 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa26c009bb0 con 0x7fa27c0fe990 2023-12-14T18:24:25.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.230+0000 7fa263fff700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2640a1150 0x7fa2640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:25.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.230+0000 7fa27a7fc700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2640a1150 0x7fa2640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:25.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.230+0000 7fa263fff700 1 -- 172.21.15.93:0/2308083500 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa26c0cc5b0 con 0x7fa27c0fe990 2023-12-14T18:24:25.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.231+0000 7fa27a7fc700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2640a1150 0x7fa2640a3600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b5c0 tx=0x7fa270005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:25.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.234+0000 7fa263fff700 1 -- 172.21.15.93:0/2308083500 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa26c0961a0 con 0x7fa27c0fe990 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: Deploying daemon mds.i on smithi160 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.97:6836/3050538896,v1:172.21.15.97:6837/3050538896] up:boot 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: fsmap 8 up:standby 2023-12-14T18:24:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:24:25.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.466+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa27c02cfe0 con 0x7fa27c0fe990 2023-12-14T18:24:25.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.467+0000 7fa263fff700 1 -- 172.21.15.93:0/2308083500 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) v1 ==== 93+0+6556 (secure 0 0 0) 0x7fa26c009940 con 0x7fa27c0fe990 2023-12-14T18:24:25.468 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:25.468 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9}],"filesystems":[]} 2023-12-14T18:24:25.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2640a1150 msgr2=0x7fa2640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:25.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2640a1150 0x7fa2640a3600 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fa27000b5c0 tx=0x7fa270005fb0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0fe990 msgr2=0x7fa27c10a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:25.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0fe990 0x7fa27c10a0e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00d980 tx=0x7fa26c00dd40 comp rx=0 tx=0).stop 2023-12-14T18:24:25.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 shutdown_connections 2023-12-14T18:24:25.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2640a1150 0x7fa2640a3600 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa27c0fe990 0x7fa27c10a0e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa27c100510 0x7fa27c108c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 --2- 172.21.15.93:0/2308083500 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa27c0ffb80 0x7fa27c108720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:25.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 >> 172.21.15.93:0/2308083500 conn(0x7fa27c0f9fb0 msgr2=0x7fa27c103090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:25.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 shutdown_connections 2023-12-14T18:24:25.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:25.469+0000 7fa281528700 1 -- 172.21.15.93:0/2308083500 wait complete. 2023-12-14T18:24:25.471 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 9 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: Deploying daemon mds.i on smithi160 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.97:6836/3050538896,v1:172.21.15.97:6837/3050538896] up:boot 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: fsmap 8 up:standby 2023-12-14T18:24:25.481 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:25 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:24:25.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:25.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:25.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:25.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: Deploying daemon mds.i on smithi160 2023-12-14T18:24:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.97:6836/3050538896,v1:172.21.15.97:6837/3050538896] up:boot 2023-12-14T18:24:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: fsmap 8 up:standby 2023-12-14T18:24:25.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:25 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:24:25.687 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:24:25 smithi160 systemd[1]: Starting Ceph mds.i for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:26.107 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:24:25 smithi160 bash[69628]: f743a390e849e70e73c4a73c7dfd72c1e49c856ecf337cc5ab6c0f5e196aba0e 2023-12-14T18:24:26.108 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:24:25 smithi160 systemd[1]: Started Ceph mds.i for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:26.108 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-mds[69666]: -- 172.21.15.160:0/4012123933 <== mon.1 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x556154581520 con 0x556154557000 2023-12-14T18:24:26.108 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:24:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-i[69643]: starting mds.i at 2023-12-14T18:24:26.368 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:26 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2308083500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:26.368 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:26.369 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:26.369 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:26.369 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:26.369 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:26.369 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:26 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:26.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:26 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2308083500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:26.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:26.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:26.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:26.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:26.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:26.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:26 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:26.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:26 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2308083500' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:26.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:26.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:26.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:26.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:26.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:26.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:26 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:27.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:27 smithi093 ceph-mon[42070]: Deploying daemon mds.g on smithi093 2023-12-14T18:24:27.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:27 smithi093 ceph-mon[42070]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:27.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:27 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] up:boot 2023-12-14T18:24:27.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:27 smithi093 ceph-mon[42070]: fsmap 9 up:standby 2023-12-14T18:24:27.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:27 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:24:27.316 INFO:journalctl@ceph.mds.g.smithi093.stdout:Dec 14 18:24:27 smithi093 systemd[1]: Starting Ceph mds.g for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:27.484 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:27.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:27 smithi160 ceph-mon[43840]: Deploying daemon mds.g on smithi093 2023-12-14T18:24:27.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:27 smithi160 ceph-mon[43840]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:27.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:27 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] up:boot 2023-12-14T18:24:27.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:27 smithi160 ceph-mon[43840]: fsmap 9 up:standby 2023-12-14T18:24:27.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:27 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:24:27.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:27 smithi097 ceph-mon[43486]: Deploying daemon mds.g on smithi093 2023-12-14T18:24:27.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:27 smithi097 ceph-mon[43486]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:27.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:27 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] up:boot 2023-12-14T18:24:27.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:27 smithi097 ceph-mon[43486]: fsmap 9 up:standby 2023-12-14T18:24:27.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:27 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:24:27.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 -- 172.21.15.93:0/3170742135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c01014b0 msgr2=0x7f49c0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:27.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 --2- 172.21.15.93:0/3170742135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c01014b0 0x7f49c0101900 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f49b0009990 tx=0x7f49b0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 -- 172.21.15.93:0/3170742135 shutdown_connections 2023-12-14T18:24:27.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 --2- 172.21.15.93:0/3170742135 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c0101e40 0x7f49c0108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 --2- 172.21.15.93:0/3170742135 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c00fe4e0 0x7f49c00fe8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 --2- 172.21.15.93:0/3170742135 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c01014b0 0x7f49c0101900 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 -- 172.21.15.93:0/3170742135 >> 172.21.15.93:0/3170742135 conn(0x7f49c00f9e80 msgr2=0x7f49c00fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:27.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 -- 172.21.15.93:0/3170742135 shutdown_connections 2023-12-14T18:24:27.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 -- 172.21.15.93:0/3170742135 wait complete. 2023-12-14T18:24:27.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 Processor -- start 2023-12-14T18:24:27.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.623+0000 7f49c86f2700 1 -- start start 2023-12-14T18:24:27.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c86f2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c00fe4e0 0x7f49c0195e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:27.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c86f2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c01014b0 0x7f49c01963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:27.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c86f2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c0101e40 0x7f49c019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:27.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c86f2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c019da10 con 0x7f49c00fe4e0 2023-12-14T18:24:27.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c86f2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c019db50 con 0x7f49c01014b0 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c86f2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c019dce0 con 0x7f49c0101e40 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c648e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c00fe4e0 0x7f49c0195e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c648e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c00fe4e0 0x7f49c0195e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35690/0 (socket says 172.21.15.93:35690) 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c648e700 1 -- 172.21.15.93:0/1599400374 learned_addr learned my addr 172.21.15.93:0/1599400374 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c6c8f700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c0101e40 0x7f49c019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c5c8d700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c01014b0 0x7f49c01963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c648e700 1 -- 172.21.15.93:0/1599400374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c0101e40 msgr2=0x7f49c019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c648e700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c0101e40 0x7f49c019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c648e700 1 -- 172.21.15.93:0/1599400374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c01014b0 msgr2=0x7f49c01963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c648e700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c01014b0 0x7f49c01963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c648e700 1 -- 172.21.15.93:0/1599400374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49b00095f0 con 0x7f49c00fe4e0 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c5c8d700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c01014b0 0x7f49c01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c648e700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c00fe4e0 0x7f49c0195e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00c990 tx=0x7f49bc00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49b77fe700 1 -- 172.21.15.93:0/1599400374 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc007b00 con 0x7f49c00fe4e0 2023-12-14T18:24:27.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49c010bb80 con 0x7f49c00fe4e0 2023-12-14T18:24:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49b77fe700 1 -- 172.21.15.93:0/1599400374 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f49bc007c60 con 0x7f49c00fe4e0 2023-12-14T18:24:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49b77fe700 1 -- 172.21.15.93:0/1599400374 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc018660 con 0x7f49c00fe4e0 2023-12-14T18:24:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.624+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c010b9f0 con 0x7f49c00fe4e0 2023-12-14T18:24:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.626+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c0107150 con 0x7f49c00fe4e0 2023-12-14T18:24:27.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.627+0000 7f49b77fe700 1 -- 172.21.15.93:0/1599400374 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f49bc00f450 con 0x7f49c00fe4e0 2023-12-14T18:24:27.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.628+0000 7f49b77fe700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f49ac0a1150 0x7f49ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:27.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.628+0000 7f49c5c8d700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f49ac0a1150 0x7f49ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:27.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.628+0000 7f49b77fe700 1 -- 172.21.15.93:0/1599400374 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f49bc0cc460 con 0x7f49c00fe4e0 2023-12-14T18:24:27.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.629+0000 7f49c5c8d700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f49ac0a1150 0x7f49ac0a3600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f49b0006010 tx=0x7f49b000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:27.632 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.632+0000 7f49b77fe700 1 -- 172.21.15.93:0/1599400374 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f49bc0960e0 con 0x7f49c00fe4e0 2023-12-14T18:24:27.814 INFO:journalctl@ceph.mds.g.smithi093.stdout:Dec 14 18:24:27 smithi093 ceph-mds[75945]: -- 172.21.15.93:0/2569013870 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x560082509520 con 0x5600824de800 2023-12-14T18:24:27.814 INFO:journalctl@ceph.mds.g.smithi093.stdout:Dec 14 18:24:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-g[75921]: starting mds.g at 2023-12-14T18:24:27.814 INFO:journalctl@ceph.mds.g.smithi093.stdout:Dec 14 18:24:27 smithi093 bash[75907]: 77cf8a4eb84cdba96ca9013ec3278c17a25daa8b144477d17cda3e39986d08be 2023-12-14T18:24:27.815 INFO:journalctl@ceph.mds.g.smithi093.stdout:Dec 14 18:24:27 smithi093 systemd[1]: Started Ceph mds.g for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:27.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.876+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49c00619b0 con 0x7f49c00fe4e0 2023-12-14T18:24:27.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.876+0000 7f49b77fe700 1 -- 172.21.15.93:0/1599400374 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) v1 ==== 94+0+7318 (secure 0 0 0) 0x7f49bc099700 con 0x7f49c00fe4e0 2023-12-14T18:24:27.877 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:27.877 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-12-14T18:24:27.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f49ac0a1150 msgr2=0x7f49ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:27.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f49ac0a1150 0x7f49ac0a3600 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f49b0006010 tx=0x7f49b000b540 comp rx=0 tx=0).stop 2023-12-14T18:24:27.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c00fe4e0 msgr2=0x7f49c0195e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c00fe4e0 0x7f49c0195e90 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f49bc00c990 tx=0x7f49bc00cca0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 shutdown_connections 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f49ac0a1150 0x7f49ac0a3600 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49c00fe4e0 0x7f49c0195e90 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49c0101e40 0x7f49c019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 --2- 172.21.15.93:0/1599400374 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49c01014b0 0x7f49c01963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.878+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 >> 172.21.15.93:0/1599400374 conn(0x7f49c00f9e80 msgr2=0x7f49c00fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.879+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 shutdown_connections 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:27.879+0000 7f49c86f2700 1 -- 172.21.15.93:0/1599400374 wait complete. 2023-12-14T18:24:27.880 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 10 2023-12-14T18:24:28.813 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:28.813 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:28.813 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:28.813 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:28.813 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:28.813 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:28.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: Deploying daemon mds.a on smithi093 2023-12-14T18:24:28.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1599400374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:28.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:28.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.93:6838/1450887105,v1:172.21.15.93:6839/1450887105] up:boot 2023-12-14T18:24:28.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: fsmap 10 up:standby 2023-12-14T18:24:28.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:28 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:24:28.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:28.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: Deploying daemon mds.a on smithi093 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1599400374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.93:6838/1450887105,v1:172.21.15.93:6839/1450887105] up:boot 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: fsmap 10 up:standby 2023-12-14T18:24:28.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:28 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: Deploying daemon mds.a on smithi093 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1599400374' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.93:6838/1450887105,v1:172.21.15.93:6839/1450887105] up:boot 2023-12-14T18:24:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: fsmap 10 up:standby 2023-12-14T18:24:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:28 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:24:29.065 INFO:journalctl@ceph.mds.a.smithi093.stdout:Dec 14 18:24:28 smithi093 systemd[1]: Starting Ceph mds.a for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:29.564 INFO:journalctl@ceph.mds.a.smithi093.stdout:Dec 14 18:24:29 smithi093 ceph-mds[76331]: -- 172.21.15.93:0/2133912936 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56324c95f520 con 0x56324c934800 2023-12-14T18:24:29.565 INFO:journalctl@ceph.mds.a.smithi093.stdout:Dec 14 18:24:29 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-a[76307]: starting mds.a at 2023-12-14T18:24:29.565 INFO:journalctl@ceph.mds.a.smithi093.stdout:Dec 14 18:24:29 smithi093 bash[76293]: dcb4bdf90ea45113f3d4db140b22548c9369ed7621d73862dd85ba0a5cba2bd7 2023-12-14T18:24:29.565 INFO:journalctl@ceph.mds.a.smithi093.stdout:Dec 14 18:24:29 smithi093 systemd[1]: Started Ceph mds.a for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:29.894 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:30.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.022+0000 7fcae92fb700 1 -- 172.21.15.93:0/3476571919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae40fe8a0 msgr2=0x7fcae40fecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:30.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.022+0000 7fcae92fb700 1 --2- 172.21.15.93:0/3476571919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae40fe8a0 0x7fcae40fecb0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009990 tx=0x7fcad8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.022+0000 7fcae92fb700 1 -- 172.21.15.93:0/3476571919 shutdown_connections 2023-12-14T18:24:30.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.022+0000 7fcae92fb700 1 --2- 172.21.15.93:0/3476571919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae40fe8a0 0x7fcae40fecb0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.022+0000 7fcae92fb700 1 --2- 172.21.15.93:0/3476571919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae4100420 0x7fcae4106ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.022+0000 7fcae92fb700 1 --2- 172.21.15.93:0/3476571919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae40ffa90 0x7fcae40ffee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.022+0000 7fcae92fb700 1 -- 172.21.15.93:0/3476571919 >> 172.21.15.93:0/3476571919 conn(0x7fcae40f9e80 msgr2=0x7fcae40fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:30.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.022+0000 7fcae92fb700 1 -- 172.21.15.93:0/3476571919 shutdown_connections 2023-12-14T18:24:30.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.022+0000 7fcae92fb700 1 -- 172.21.15.93:0/3476571919 wait complete. 2023-12-14T18:24:30.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.023+0000 7fcae92fb700 1 Processor -- start 2023-12-14T18:24:30.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.023+0000 7fcae92fb700 1 -- start start 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.023+0000 7fcae92fb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae40fe8a0 0x7fcae4102830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.023+0000 7fcae92fb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae40ffa90 0x7fcae4100e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.023+0000 7fcae92fb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae4100420 0x7fcae41013b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.023+0000 7fcae92fb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae4101b50 con 0x7fcae4100420 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.023+0000 7fcae92fb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae4101cc0 con 0x7fcae40ffa90 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.023+0000 7fcae92fb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae41a0030 con 0x7fcae40fe8a0 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae4100420 0x7fcae41013b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae4100420 0x7fcae41013b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35722/0 (socket says 172.21.15.93:35722) 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae37fe700 1 -- 172.21.15.93:0/4206587995 learned_addr learned my addr 172.21.15.93:0/4206587995 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae27fc700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae40ffa90 0x7fcae4100e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae37fe700 1 -- 172.21.15.93:0/4206587995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae40fe8a0 msgr2=0x7fcae4102830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae2ffd700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae40fe8a0 0x7fcae4102830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:30.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae37fe700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae40fe8a0 0x7fcae4102830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae37fe700 1 -- 172.21.15.93:0/4206587995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae40ffa90 msgr2=0x7fcae4100e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae37fe700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae40ffa90 0x7fcae4100e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae37fe700 1 -- 172.21.15.93:0/4206587995 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80095f0 con 0x7fcae4100420 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae27fc700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae40ffa90 0x7fcae4100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae37fe700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae4100420 0x7fcae41013b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcadc007ee0 tx=0x7fcadc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcacbfff700 1 -- 172.21.15.93:0/4206587995 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc017070 con 0x7fcae4100420 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcacbfff700 1 -- 172.21.15.93:0/4206587995 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcadc00f040 con 0x7fcae4100420 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae410aa90 con 0x7fcae4100420 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcacbfff700 1 -- 172.21.15.93:0/4206587995 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcadc005740 con 0x7fcae4100420 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.024+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae410a900 con 0x7fcae4100420 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.025+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae41a06d0 con 0x7fcae4100420 2023-12-14T18:24:30.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.026+0000 7fcacbfff700 1 -- 172.21.15.93:0/4206587995 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcadc0058a0 con 0x7fcae4100420 2023-12-14T18:24:30.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.027+0000 7fcacbfff700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcacc0a1150 0x7fcacc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:30.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.027+0000 7fcae2ffd700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcacc0a1150 0x7fcacc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:30.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.027+0000 7fcacbfff700 1 -- 172.21.15.93:0/4206587995 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fcadc0cbd30 con 0x7fcae4100420 2023-12-14T18:24:30.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.027+0000 7fcae2ffd700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcacc0a1150 0x7fcacc0a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcad80095c0 tx=0x7fcad8009530 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:30.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.031+0000 7fcacbfff700 1 -- 172.21.15.93:0/4206587995 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcadc095920 con 0x7fcae4100420 2023-12-14T18:24:30.263 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.262+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcae41a0c80 con 0x7fcae4100420 2023-12-14T18:24:30.264 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.263+0000 7fcacbfff700 1 -- 172.21.15.93:0/4206587995 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) v1 ==== 94+0+8834 (secure 0 0 0) 0x7fcadc098f40 con 0x7fcae4100420 2023-12-14T18:24:30.264 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:30.264 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[]} 2023-12-14T18:24:30.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcacc0a1150 msgr2=0x7fcacc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:30.266 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcacc0a1150 0x7fcacc0a3600 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fcad80095c0 tx=0x7fcad8009530 comp rx=0 tx=0).stop 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae4100420 msgr2=0x7fcae41013b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae4100420 0x7fcae41013b0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fcadc007ee0 tx=0x7fcadc00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 shutdown_connections 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcacc0a1150 0x7fcacc0a3600 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae4100420 0x7fcae41013b0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae40fe8a0 0x7fcae4102830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 --2- 172.21.15.93:0/4206587995 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae40ffa90 0x7fcae4100e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 >> 172.21.15.93:0/4206587995 conn(0x7fcae40f9e80 msgr2=0x7fcae4107380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 shutdown_connections 2023-12-14T18:24:30.267 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:30.265+0000 7fcae92fb700 1 -- 172.21.15.93:0/4206587995 wait complete. 2023-12-14T18:24:30.268 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 12 2023-12-14T18:24:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: Deploying daemon mds.b on smithi097 2023-12-14T18:24:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.93:6840/4265710188,v1:172.21.15.93:6841/4265710188] up:boot 2023-12-14T18:24:30.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: fsmap 11 up:standby 2023-12-14T18:24:30.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:24:30.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:30.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:30 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4206587995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:30.584 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:30.584 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:30.584 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:30.584 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:30.584 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:30.584 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:30.584 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: Deploying daemon mds.b on smithi097 2023-12-14T18:24:30.584 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.93:6840/4265710188,v1:172.21.15.93:6841/4265710188] up:boot 2023-12-14T18:24:30.585 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: fsmap 11 up:standby 2023-12-14T18:24:30.585 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:24:30.585 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:30.585 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:30 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4206587995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2023-12-14T18:24:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2023-12-14T18:24:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: Deploying daemon mds.b on smithi097 2023-12-14T18:24:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.93:6840/4265710188,v1:172.21.15.93:6841/4265710188] up:boot 2023-12-14T18:24:30.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: fsmap 11 up:standby 2023-12-14T18:24:30.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:24:30.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: pgmap v213: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:30.747 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:30 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4206587995' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:30.843 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:24:30 smithi097 systemd[1]: Starting Ceph mds.b for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:24:31.343 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:24:31 smithi097 ceph-mds[68583]: -- 172.21.15.97:0/250196458 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5623c16ad520 con 0x5623c1682800 2023-12-14T18:24:31.343 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:24:31 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-b[68560]: starting mds.b at 2023-12-14T18:24:31.343 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:24:31 smithi097 bash[68545]: 979b27db1e779c44613360294ca928890526bd9ee795f711b5ee029a15594083 2023-12-14T18:24:31.343 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:24:31 smithi097 systemd[1]: Started Ceph mds.b for 1c5ac00e-9aad-11ee-95a5-87774f69a715. 2023-12-14T18:24:32.280 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:32.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 -- 172.21.15.93:0/2392622430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec103600 msgr2=0x7f72ec1059e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:32.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2392622430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec103600 0x7f72ec1059e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f72e0009990 tx=0x7f72e0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 -- 172.21.15.93:0/2392622430 shutdown_connections 2023-12-14T18:24:32.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2392622430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec105f20 0x7f72ec108380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2392622430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec100ce0 0x7f72ec1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2392622430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec103600 0x7f72ec1059e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 -- 172.21.15.93:0/2392622430 >> 172.21.15.93:0/2392622430 conn(0x7f72ec0fa740 msgr2=0x7f72ec0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:32.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 -- 172.21.15.93:0/2392622430 shutdown_connections 2023-12-14T18:24:32.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 -- 172.21.15.93:0/2392622430 wait complete. 2023-12-14T18:24:32.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.418+0000 7f72f21cf700 1 Processor -- start 2023-12-14T18:24:32.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72f21cf700 1 -- start start 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72f21cf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec100ce0 0x7f72ec100650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72f21cf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec103600 0x7f72ec0fec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72f21cf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec105f20 0x7f72ec0ff1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72f21cf700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec0ff970 con 0x7f72ec103600 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72f21cf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec0ffae0 con 0x7f72ec105f20 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72f21cf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72ec1a0000 con 0x7f72ec100ce0 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72eb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec100ce0 0x7f72ec100650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72eb7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec100ce0 0x7f72ec100650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:50116/0 (socket says 172.21.15.93:50116) 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72eb7fe700 1 -- 172.21.15.93:0/2193715074 learned_addr learned my addr 172.21.15.93:0/2193715074 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72eaffd700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec103600 0x7f72ec0fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72ebfff700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec105f20 0x7f72ec0ff1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72eb7fe700 1 -- 172.21.15.93:0/2193715074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec105f20 msgr2=0x7f72ec0ff1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:32.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72eb7fe700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec105f20 0x7f72ec0ff1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72eb7fe700 1 -- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec103600 msgr2=0x7f72ec0fec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72eb7fe700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec103600 0x7f72ec0fec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72eb7fe700 1 -- 172.21.15.93:0/2193715074 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72e00095f0 con 0x7f72ec100ce0 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.419+0000 7f72ebfff700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec105f20 0x7f72ec0ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.420+0000 7f72eb7fe700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec100ce0 0x7f72ec100650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00d980 tx=0x7f72dc00dc90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.420+0000 7f72e8ff9700 1 -- 172.21.15.93:0/2193715074 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc004220 con 0x7f72ec100ce0 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.420+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72ec10bc30 con 0x7f72ec100ce0 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.420+0000 7f72e8ff9700 1 -- 172.21.15.93:0/2193715074 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f72dc007640 con 0x7f72ec100ce0 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.420+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72ec10baa0 con 0x7f72ec100ce0 2023-12-14T18:24:32.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.420+0000 7f72e8ff9700 1 -- 172.21.15.93:0/2193715074 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72dc005690 con 0x7f72ec100ce0 2023-12-14T18:24:32.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.421+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72ec067160 con 0x7f72ec100ce0 2023-12-14T18:24:32.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.422+0000 7f72e8ff9700 1 -- 172.21.15.93:0/2193715074 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f72dc004750 con 0x7f72ec100ce0 2023-12-14T18:24:32.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.423+0000 7f72e8ff9700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f72d40a1150 0x7f72d40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:32.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.423+0000 7f72eaffd700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f72d40a1150 0x7f72d40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:32.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.423+0000 7f72e8ff9700 1 -- 172.21.15.93:0/2193715074 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f72dc0cc2e0 con 0x7f72ec100ce0 2023-12-14T18:24:32.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.423+0000 7f72eaffd700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f72d40a1150 0x7f72d40a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f72e000b2e0 tx=0x7f72e0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:32.428 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.427+0000 7f72e8ff9700 1 -- 172.21.15.93:0/2193715074 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f72dc095ed0 con 0x7f72ec100ce0 2023-12-14T18:24:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:32 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] up:boot 2023-12-14T18:24:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:32 smithi160 ceph-mon[43840]: fsmap 12 up:standby 2023-12-14T18:24:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:32 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:24:32.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:32 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] up:boot 2023-12-14T18:24:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:32 smithi093 ceph-mon[42070]: fsmap 12 up:standby 2023-12-14T18:24:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:32 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:24:32.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:32 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] up:boot 2023-12-14T18:24:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:32 smithi097 ceph-mon[43486]: fsmap 12 up:standby 2023-12-14T18:24:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:32 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:24:32.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.701+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72ec1a0c20 con 0x7f72ec100ce0 2023-12-14T18:24:32.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.702+0000 7f72e8ff9700 1 -- 172.21.15.93:0/2193715074 <== mon.1 v2:172.21.15.160:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) v1 ==== 94+0+9592 (secure 0 0 0) 0x7f72dc0994f0 con 0x7f72ec100ce0 2023-12-14T18:24:32.703 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:32.703 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10},{"gid":24461,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":13}],"filesystems":[]} 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f72d40a1150 msgr2=0x7f72d40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f72d40a1150 0x7f72d40a3600 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f72e000b2e0 tx=0x7f72e0005fb0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec100ce0 msgr2=0x7f72ec100650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec100ce0 0x7f72ec100650 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f72dc00d980 tx=0x7f72dc00dc90 comp rx=0 tx=0).stop 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 shutdown_connections 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f72d40a1150 0x7f72d40a3600 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72ec105f20 0x7f72ec0ff1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72ec100ce0 0x7f72ec100650 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 --2- 172.21.15.93:0/2193715074 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f72ec103600 0x7f72ec0fec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.704+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 >> 172.21.15.93:0/2193715074 conn(0x7f72ec0fa740 msgr2=0x7f72ec102140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:32.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.705+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 shutdown_connections 2023-12-14T18:24:32.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.705+0000 7f72f21cf700 1 -- 172.21.15.93:0/2193715074 wait complete. 2023-12-14T18:24:32.707 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 13 2023-12-14T18:24:32.718 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:24:32.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.841+0000 7fd0c88c5700 1 -- 172.21.15.93:0/3217160708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 msgr2=0x7fd0c4108330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:32.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.841+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/3217160708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 0x7fd0c4108330 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc00b0a0 tx=0x7fd0bc00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.841+0000 7fd0c88c5700 1 -- 172.21.15.93:0/3217160708 shutdown_connections 2023-12-14T18:24:32.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.841+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/3217160708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 0x7fd0c4108330 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.841+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/3217160708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c4100c90 0x7fd0c4103070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.841+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/3217160708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c41035b0 0x7fd0c4105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.841+0000 7fd0c88c5700 1 -- 172.21.15.93:0/3217160708 >> 172.21.15.93:0/3217160708 conn(0x7fd0c40fa6f0 msgr2=0x7fd0c40fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:32.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.841+0000 7fd0c88c5700 1 -- 172.21.15.93:0/3217160708 shutdown_connections 2023-12-14T18:24:32.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.841+0000 7fd0c88c5700 1 -- 172.21.15.93:0/3217160708 wait complete. 2023-12-14T18:24:32.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c88c5700 1 Processor -- start 2023-12-14T18:24:32.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c88c5700 1 -- start start 2023-12-14T18:24:32.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c88c5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c4100c90 0x7fd0c4196060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:32.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c88c5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c41035b0 0x7fd0c41965a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:32.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c88c5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 0x7fd0c419d610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:32.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c88c5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c419dbe0 con 0x7fd0c4105ed0 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c88c5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c419dd20 con 0x7fd0c41035b0 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c88c5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0c419deb0 con 0x7fd0c4100c90 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 0x7fd0c419d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c4100c90 0x7fd0c4196060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c2d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 0x7fd0c419d610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56378/0 (socket says 172.21.15.93:56378) 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c2d9d700 1 -- 172.21.15.93:0/1692022301 learned_addr learned my addr 172.21.15.93:0/1692022301 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c1d9b700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c41035b0 0x7fd0c41965a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c2d9d700 1 -- 172.21.15.93:0/1692022301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c4100c90 msgr2=0x7fd0c4196060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c2d9d700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c4100c90 0x7fd0c4196060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c2d9d700 1 -- 172.21.15.93:0/1692022301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c41035b0 msgr2=0x7fd0c41965a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c2d9d700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c41035b0 0x7fd0c41965a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:32.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c2d9d700 1 -- 172.21.15.93:0/1692022301 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0bc009d00 con 0x7fd0c4105ed0 2023-12-14T18:24:32.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.842+0000 7fd0c2d9d700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 0x7fd0c419d610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0062a0 tx=0x7fd0bc004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:32.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.843+0000 7fd0b37fe700 1 -- 172.21.15.93:0/1692022301 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc00e040 con 0x7fd0c4105ed0 2023-12-14T18:24:32.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.843+0000 7fd0b37fe700 1 -- 172.21.15.93:0/1692022301 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0bc00cec0 con 0x7fd0c4105ed0 2023-12-14T18:24:32.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.843+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0c410bb70 con 0x7fd0c4105ed0 2023-12-14T18:24:32.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.843+0000 7fd0b37fe700 1 -- 172.21.15.93:0/1692022301 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0bc01baf0 con 0x7fd0c4105ed0 2023-12-14T18:24:32.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.843+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0c410b9e0 con 0x7fd0c4105ed0 2023-12-14T18:24:32.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.844+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0c40fc260 con 0x7fd0c4105ed0 2023-12-14T18:24:32.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.844+0000 7fd0b37fe700 1 -- 172.21.15.93:0/1692022301 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd0bc01bc50 con 0x7fd0c4105ed0 2023-12-14T18:24:32.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.845+0000 7fd0b37fe700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0ac0a11d0 0x7fd0ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:32.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.845+0000 7fd0c259c700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0ac0a11d0 0x7fd0ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:32.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.845+0000 7fd0b37fe700 1 -- 172.21.15.93:0/1692022301 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fd0bc0cdf40 con 0x7fd0c4105ed0 2023-12-14T18:24:32.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.845+0000 7fd0c259c700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0ac0a11d0 0x7fd0ac0a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8007c60 tx=0x7fd0b80073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:32.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:32.849+0000 7fd0b37fe700 1 -- 172.21.15.93:0/1692022301 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd0bc097bc0 con 0x7fd0c4105ed0 2023-12-14T18:24:33.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.045+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0c402cfe0 con 0x7fd0c4105ed0 2023-12-14T18:24:33.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.046+0000 7fd0b37fe700 1 -- 172.21.15.93:0/1692022301 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v76) v1 ==== 74+0+18350 (secure 0 0 0) 0x7fd0bc09b1e0 con 0x7fd0c4105ed0 2023-12-14T18:24:33.048 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:33.048 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":76,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:04.133253+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":50,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":71,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:33.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.048+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0ac0a11d0 msgr2=0x7fd0ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.048+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0ac0a11d0 0x7fd0ac0a3680 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd0b8007c60 tx=0x7fd0b80073d0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.048+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 msgr2=0x7fd0c419d610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.048+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 0x7fd0c419d610 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fd0bc0062a0 tx=0x7fd0bc004d00 comp rx=0 tx=0).stop 2023-12-14T18:24:33.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.048+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 shutdown_connections 2023-12-14T18:24:33.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.048+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0ac0a11d0 0x7fd0ac0a3680 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.048+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0c4105ed0 0x7fd0c419d610 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.048+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0c4100c90 0x7fd0c4196060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.048+0000 7fd0c88c5700 1 --2- 172.21.15.93:0/1692022301 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0c41035b0 0x7fd0c41965a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.049+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 >> 172.21.15.93:0/1692022301 conn(0x7fd0c40fa6f0 msgr2=0x7fd0c41018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:33.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.049+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 shutdown_connections 2023-12-14T18:24:33.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.049+0000 7fd0c88c5700 1 -- 172.21.15.93:0/1692022301 wait complete. 2023-12-14T18:24:33.062 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-14T18:24:33.062 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:24:33.197 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 -- 172.21.15.93:0/2753833170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2701014a0 msgr2=0x7fa2701018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 --2- 172.21.15.93:0/2753833170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2701014a0 0x7fa2701018f0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fa260009990 tx=0x7fa260009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 -- 172.21.15.93:0/2753833170 shutdown_connections 2023-12-14T18:24:33.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 --2- 172.21.15.93:0/2753833170 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa270101e30 0x7fa2701086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 --2- 172.21.15.93:0/2753833170 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2700fe4d0 0x7fa2700fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 --2- 172.21.15.93:0/2753833170 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2701014a0 0x7fa2701018f0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 -- 172.21.15.93:0/2753833170 >> 172.21.15.93:0/2753833170 conn(0x7fa2700f9ed0 msgr2=0x7fa2700fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:33.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 -- 172.21.15.93:0/2753833170 shutdown_connections 2023-12-14T18:24:33.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 -- 172.21.15.93:0/2753833170 wait complete. 2023-12-14T18:24:33.198 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 Processor -- start 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.194+0000 7fa2783e4700 1 -- start start 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2783e4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2700fe4d0 0x7fa270100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2783e4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2701014a0 0x7fa2700ff450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2783e4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa270101e30 0x7fa2700ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2783e4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa270100130 con 0x7fa2700fe4d0 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2783e4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2701002a0 con 0x7fa2701014a0 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2783e4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa27019ff60 con 0x7fa270101e30 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276180700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2700fe4d0 0x7fa270100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276180700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2700fe4d0 0x7fa270100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56398/0 (socket says 172.21.15.93:56398) 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276180700 1 -- 172.21.15.93:0/3054588793 learned_addr learned my addr 172.21.15.93:0/3054588793 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa27597f700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2701014a0 0x7fa2700ff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.199 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276981700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa270101e30 0x7fa2700ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276180700 1 -- 172.21.15.93:0/3054588793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa270101e30 msgr2=0x7fa2700ff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276180700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa270101e30 0x7fa2700ff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276180700 1 -- 172.21.15.93:0/3054588793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2701014a0 msgr2=0x7fa2700ff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276180700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2701014a0 0x7fa2700ff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276180700 1 -- 172.21.15.93:0/3054588793 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2600095f0 con 0x7fa2700fe4d0 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa276180700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2700fe4d0 0x7fa270100e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00c990 tx=0x7fa26c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2677fe700 1 -- 172.21.15.93:0/3054588793 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c007b00 con 0x7fa2700fe4d0 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2677fe700 1 -- 172.21.15.93:0/3054588793 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa26c007c60 con 0x7fa2700fe4d0 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa27010bb70 con 0x7fa2700fe4d0 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2677fe700 1 -- 172.21.15.93:0/3054588793 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c018660 con 0x7fa2700fe4d0 2023-12-14T18:24:33.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.195+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa27010b9e0 con 0x7fa2700fe4d0 2023-12-14T18:24:33.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.196+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2701071e0 con 0x7fa2700fe4d0 2023-12-14T18:24:33.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.197+0000 7fa2677fe700 1 -- 172.21.15.93:0/3054588793 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa26c00f450 con 0x7fa2700fe4d0 2023-12-14T18:24:33.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.198+0000 7fa2677fe700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa25c0a1150 0x7fa25c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.198+0000 7fa27597f700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa25c0a1150 0x7fa25c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.198+0000 7fa2677fe700 1 -- 172.21.15.93:0/3054588793 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7fa26c0cc3e0 con 0x7fa2700fe4d0 2023-12-14T18:24:33.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.198+0000 7fa27597f700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa25c0a1150 0x7fa25c0a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa26000b5c0 tx=0x7fa260005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:33.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.202+0000 7fa2677fe700 1 -- 172.21.15.93:0/3054588793 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa26c095fd0 con 0x7fa2700fe4d0 2023-12-14T18:24:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2193715074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1692022301' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:33 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.403+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa2700619b0 con 0x7fa2700fe4d0 2023-12-14T18:24:33.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.404+0000 7fa2677fe700 1 -- 172.21.15.93:0/3054588793 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v76) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa26c0995f0 con 0x7fa2700fe4d0 2023-12-14T18:24:33.405 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:24:33.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa25c0a1150 msgr2=0x7fa25c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa25c0a1150 0x7fa25c0a3600 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa26000b5c0 tx=0x7fa260005960 comp rx=0 tx=0).stop 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2700fe4d0 msgr2=0x7fa270100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2700fe4d0 0x7fa270100e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa26c00c990 tx=0x7fa26c00cca0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 shutdown_connections 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa25c0a1150 0x7fa25c0a3600 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2700fe4d0 0x7fa270100e10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa270101e30 0x7fa2700ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 --2- 172.21.15.93:0/3054588793 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2701014a0 0x7fa2700ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 >> 172.21.15.93:0/3054588793 conn(0x7fa2700f9ed0 msgr2=0x7fa2700fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 shutdown_connections 2023-12-14T18:24:33.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.406+0000 7fa2783e4700 1 -- 172.21.15.93:0/3054588793 wait complete. 2023-12-14T18:24:33.419 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-14T18:24:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2193715074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1692022301' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:33 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.544+0000 7f43faa8e700 1 -- 172.21.15.93:0/1503090394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43f41013e0 msgr2=0x7f43f4103860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.544+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1503090394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43f41013e0 0x7f43f4103860 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f43e0009990 tx=0x7f43e0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.544+0000 7f43faa8e700 1 -- 172.21.15.93:0/1503090394 shutdown_connections 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.544+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1503090394 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43f4103da0 0x7f43f4104210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.544+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1503090394 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43f40fea80 0x7f43f4100ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.544+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1503090394 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43f41013e0 0x7f43f4103860 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.544+0000 7f43faa8e700 1 -- 172.21.15.93:0/1503090394 >> 172.21.15.93:0/1503090394 conn(0x7f43f40fa6f0 msgr2=0x7f43f40fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.544+0000 7f43faa8e700 1 -- 172.21.15.93:0/1503090394 shutdown_connections 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.544+0000 7f43faa8e700 1 -- 172.21.15.93:0/1503090394 wait complete. 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43faa8e700 1 Processor -- start 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43faa8e700 1 -- start start 2023-12-14T18:24:33.548 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43faa8e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43f40fea80 0x7f43f419c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43faa8e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43f41013e0 0x7f43f419cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43faa8e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43f4103da0 0x7f43f41a3b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43faa8e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f41a4120 con 0x7f43f40fea80 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43faa8e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f41a4260 con 0x7f43f4103da0 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43faa8e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43f41a43f0 con 0x7f43f41013e0 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43f41013e0 0x7f43f419cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f882a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43f40fea80 0x7f43f419c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f3fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43f41013e0 0x7f43f419cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:50176/0 (socket says 172.21.15.93:50176) 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f3fff700 1 -- 172.21.15.93:0/1912497876 learned_addr learned my addr 172.21.15.93:0/1912497876 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:33.549 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f902b700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43f4103da0 0x7f43f41a3b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f882a700 1 -- 172.21.15.93:0/1912497876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43f41013e0 msgr2=0x7f43f419cae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f882a700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43f41013e0 0x7f43f419cae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f882a700 1 -- 172.21.15.93:0/1912497876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43f4103da0 msgr2=0x7f43f41a3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f882a700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43f4103da0 0x7f43f41a3b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f882a700 1 -- 172.21.15.93:0/1912497876 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43e00095f0 con 0x7f43f40fea80 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.545+0000 7f43f3fff700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43f41013e0 0x7f43f419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.546+0000 7f43f882a700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43f40fea80 0x7f43f419c5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e9f0 tx=0x7f43e800ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.546+0000 7f43f1ffb700 1 -- 172.21.15.93:0/1912497876 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e800c9c0 con 0x7f43f40fea80 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.546+0000 7f43f1ffb700 1 -- 172.21.15.93:0/1912497876 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f43e8010430 con 0x7f43f40fea80 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.546+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43f4112170 con 0x7f43f40fea80 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.546+0000 7f43f1ffb700 1 -- 172.21.15.93:0/1912497876 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43e8018680 con 0x7f43f40fea80 2023-12-14T18:24:33.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.546+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43f4111fe0 con 0x7f43f40fea80 2023-12-14T18:24:33.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.547+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43f40fc260 con 0x7f43f40fea80 2023-12-14T18:24:33.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.548+0000 7f43f1ffb700 1 -- 172.21.15.93:0/1912497876 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f43e8010860 con 0x7f43f40fea80 2023-12-14T18:24:33.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.548+0000 7f43f1ffb700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f43e40a1150 0x7f43e40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.549+0000 7f43f1ffb700 1 -- 172.21.15.93:0/1912497876 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f43e8014070 con 0x7f43f40fea80 2023-12-14T18:24:33.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.549+0000 7f43f3fff700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f43e40a1150 0x7f43e40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.549+0000 7f43f3fff700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f43e40a1150 0x7f43e40a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43e000b2e0 tx=0x7f43e0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:33.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.553+0000 7f43f1ffb700 1 -- 172.21.15.93:0/1912497876 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f43e8051730 con 0x7f43f40fea80 2023-12-14T18:24:33.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2193715074' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1692022301' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:33.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:33 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:33.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.798+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f43f4061950 con 0x7f43f40fea80 2023-12-14T18:24:33.799 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.798+0000 7f43f1ffb700 1 -- 172.21.15.93:0/1912497876 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v76) v1 ==== 84+0+65 (secure 0 0 0) 0x7f43e8096180 con 0x7f43f40fea80 2023-12-14T18:24:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:33.799 INFO:teuthology.orchestra.run.smithi093.stdout:[ 2023-12-14T18:24:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:24:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "poolnum": 1, 2023-12-14T18:24:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "poolname": ".mgr" 2023-12-14T18:24:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:24:33.800 INFO:teuthology.orchestra.run.smithi093.stdout:] 2023-12-14T18:24:33.801 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f43e40a1150 msgr2=0x7f43e40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f43e40a1150 0x7f43e40a3600 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f43e000b2e0 tx=0x7f43e0005960 comp rx=0 tx=0).stop 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43f40fea80 msgr2=0x7f43f419c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43f40fea80 0x7f43f419c5a0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f43e800e9f0 tx=0x7f43e800ed00 comp rx=0 tx=0).stop 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 shutdown_connections 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f43e40a1150 0x7f43e40a3600 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f43f40fea80 0x7f43f419c5a0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f43f4103da0 0x7f43f41a3b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 --2- 172.21.15.93:0/1912497876 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f43f41013e0 0x7f43f419cae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 >> 172.21.15.93:0/1912497876 conn(0x7f43f40fa6f0 msgr2=0x7f43f40ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 shutdown_connections 2023-12-14T18:24:33.802 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.800+0000 7f43faa8e700 1 -- 172.21.15.93:0/1912497876 wait complete. 2023-12-14T18:24:33.813 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-14T18:24:33.813 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-14T18:24:33.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 -- 172.21.15.93:0/67039180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ff480 msgr2=0x7f69480ff8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 --2- 172.21.15.93:0/67039180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ff480 0x7f69480ff8f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6934009990 tx=0x7f6934009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 -- 172.21.15.93:0/67039180 shutdown_connections 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 --2- 172.21.15.93:0/67039180 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69480ffe30 0x7f69481002a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 --2- 172.21.15.93:0/67039180 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69480feb30 0x7f69480fef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 --2- 172.21.15.93:0/67039180 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ff480 0x7f69480ff8f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 -- 172.21.15.93:0/67039180 >> 172.21.15.93:0/67039180 conn(0x7f69480fa740 msgr2=0x7f69480fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 -- 172.21.15.93:0/67039180 shutdown_connections 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 -- 172.21.15.93:0/67039180 wait complete. 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 Processor -- start 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 -- start start 2023-12-14T18:24:33.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69480feb30 0x7f69481981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.947+0000 7f694eeb3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69480ff480 0x7f6948198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f694eeb3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ffe30 0x7f694819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f694eeb3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694819fd40 con 0x7f69480ffe30 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f694eeb3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f694819fe80 con 0x7f69480ff480 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f694eeb3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69481a0010 con 0x7f69480feb30 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f694d450700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ffe30 0x7f694819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f694cc4f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69480feb30 0x7f69481981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f6947fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69480ff480 0x7f6948198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f694d450700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ffe30 0x7f694819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56418/0 (socket says 172.21.15.93:56418) 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f694d450700 1 -- 172.21.15.93:0/3489910721 learned_addr learned my addr 172.21.15.93:0/3489910721 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:33.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f6947fff700 1 -- 172.21.15.93:0/3489910721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69480feb30 msgr2=0x7f69481981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f6947fff700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69480feb30 0x7f69481981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f6947fff700 1 -- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ffe30 msgr2=0x7f694819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f6947fff700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ffe30 0x7f694819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f6947fff700 1 -- 172.21.15.93:0/3489910721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69340095f0 con 0x7f69480ff480 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f694d450700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ffe30 0x7f694819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f6947fff700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69480ff480 0x7f6948198700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6934009990 tx=0x7f6934009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.948+0000 7f6945ffb700 1 -- 172.21.15.93:0/3489910721 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f693401d070 con 0x7f69480ff480 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.949+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f694810ddf0 con 0x7f69480ff480 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.949+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f694810dc60 con 0x7f69480ff480 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.949+0000 7f6945ffb700 1 -- 172.21.15.93:0/3489910721 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6934003f60 con 0x7f69480ff480 2023-12-14T18:24:33.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.949+0000 7f6945ffb700 1 -- 172.21.15.93:0/3489910721 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6934022ba0 con 0x7f69480ff480 2023-12-14T18:24:33.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.950+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f692c005320 con 0x7f69480ff480 2023-12-14T18:24:33.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.951+0000 7f6945ffb700 1 -- 172.21.15.93:0/3489910721 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f69340040d0 con 0x7f69480ff480 2023-12-14T18:24:33.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.951+0000 7f6945ffb700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f69380a1150 0x7f69380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:33.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.951+0000 7f6945ffb700 1 -- 172.21.15.93:0/3489910721 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 7127+0+0 (secure 0 0 0) 0x7f69340cda40 con 0x7f69480ff480 2023-12-14T18:24:33.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.951+0000 7f694cc4f700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f69380a1150 0x7f69380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:33.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.952+0000 7f694cc4f700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f69380a1150 0x7f69380a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f693c005950 tx=0x7f693c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:33.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:33.955+0000 7f6945ffb700 1 -- 172.21.15.93:0/3489910721 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6934097620 con 0x7f69480ff480 2023-12-14T18:24:34.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:34.158+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f692c005d60 con 0x7f69480ff480 2023-12-14T18:24:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:34 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:24:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:34 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:24:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:34 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:24:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:34 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:24:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:34 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:34 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3054588793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:34 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1912497876' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-14T18:24:34.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:34 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:24:34.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:34 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:24:34.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:34 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:24:34.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:34 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:24:34.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:34 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:34.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:34 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3054588793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:34.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:34 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1912497876' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-14T18:24:34.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:34 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:24:34.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:34 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:24:34.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:34 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:24:34.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:34 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:24:34.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:34 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:24:34.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:34 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3054588793' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:34.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:34 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1912497876' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2023-12-14T18:24:35.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.153+0000 7f6945ffb700 1 -- 172.21.15.93:0/3489910721 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6934003aa0 con 0x7f69480ff480 2023-12-14T18:24:35.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f69380a1150 msgr2=0x7f69380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:35.158 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f69380a1150 0x7f69380a3600 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f693c005950 tx=0x7f693c00a300 comp rx=0 tx=0).stop 2023-12-14T18:24:35.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69480ff480 msgr2=0x7f6948198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:35.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69480ff480 0x7f6948198700 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6934009990 tx=0x7f6934009580 comp rx=0 tx=0).stop 2023-12-14T18:24:35.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 shutdown_connections 2023-12-14T18:24:35.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f69380a1150 0x7f69380a3600 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f69480ffe30 0x7f694819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f69480feb30 0x7f69481981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 --2- 172.21.15.93:0/3489910721 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f69480ff480 0x7f6948198700 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 >> 172.21.15.93:0/3489910721 conn(0x7f69480fa740 msgr2=0x7f69480fc400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:35.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 shutdown_connections 2023-12-14T18:24:35.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.157+0000 7f694eeb3700 1 -- 172.21.15.93:0/3489910721 wait complete. 2023-12-14T18:24:35.160 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_metadata' created 2023-12-14T18:24:35.172 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2023-12-14T18:24:35.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.301+0000 7f2d86e89700 1 -- 172.21.15.93:0/3246460385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff3f0 msgr2=0x7f2d800ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:35.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.301+0000 7f2d86e89700 1 --2- 172.21.15.93:0/3246460385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff3f0 0x7f2d800ff840 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c009990 tx=0x7f2d6c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.301+0000 7f2d86e89700 1 -- 172.21.15.93:0/3246460385 shutdown_connections 2023-12-14T18:24:35.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.301+0000 7f2d86e89700 1 --2- 172.21.15.93:0/3246460385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d800ffd80 0x7f2d80107650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.301+0000 7f2d86e89700 1 --2- 172.21.15.93:0/3246460385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d801006c0 0x7f2d80100ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.306 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.301+0000 7f2d86e89700 1 --2- 172.21.15.93:0/3246460385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d800ff3f0 0x7f2d800ff840 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.301+0000 7f2d86e89700 1 -- 172.21.15.93:0/3246460385 >> 172.21.15.93:0/3246460385 conn(0x7f2d800f9eb0 msgr2=0x7f2d800fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.302+0000 7f2d86e89700 1 -- 172.21.15.93:0/3246460385 shutdown_connections 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.302+0000 7f2d86e89700 1 -- 172.21.15.93:0/3246460385 wait complete. 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.302+0000 7f2d86e89700 1 Processor -- start 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.302+0000 7f2d86e89700 1 -- start start 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.302+0000 7f2d86e89700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d800ff3f0 0x7f2d80194e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d86e89700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d800ffd80 0x7f2d80195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d86e89700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d801006c0 0x7f2d8019c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d86e89700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019c9c0 con 0x7f2d801006c0 2023-12-14T18:24:35.307 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d86e89700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019cb00 con 0x7f2d800ffd80 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d86e89700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d8019cc90 con 0x7f2d800ff3f0 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d85426700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d801006c0 0x7f2d8019c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d85426700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d801006c0 0x7f2d8019c3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56422/0 (socket says 172.21.15.93:56422) 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d85426700 1 -- 172.21.15.93:0/1958718471 learned_addr learned my addr 172.21.15.93:0/1958718471 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d7ffff700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d800ffd80 0x7f2d80195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d84c25700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d800ff3f0 0x7f2d80194e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d85426700 1 -- 172.21.15.93:0/1958718471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d800ff3f0 msgr2=0x7f2d80194e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d85426700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d800ff3f0 0x7f2d80194e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d85426700 1 -- 172.21.15.93:0/1958718471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d800ffd80 msgr2=0x7f2d80195380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d85426700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d800ffd80 0x7f2d80195380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d85426700 1 -- 172.21.15.93:0/1958718471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009c40 con 0x7f2d801006c0 2023-12-14T18:24:35.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d85426700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d801006c0 0x7f2d8019c3f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800e5a0 tx=0x7f2d7800e8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:35.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d7dffb700 1 -- 172.21.15.93:0/1958718471 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78003a00 con 0x7f2d801006c0 2023-12-14T18:24:35.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d6c0095f0 con 0x7f2d801006c0 2023-12-14T18:24:35.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d7dffb700 1 -- 172.21.15.93:0/1958718471 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d78003b60 con 0x7f2d801006c0 2023-12-14T18:24:35.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d7dffb700 1 -- 172.21.15.93:0/1958718471 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780077f0 con 0x7f2d801006c0 2023-12-14T18:24:35.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.303+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d8010aa80 con 0x7f2d801006c0 2023-12-14T18:24:35.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.304+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d80067e40 con 0x7f2d801006c0 2023-12-14T18:24:35.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.306+0000 7f2d7dffb700 1 -- 172.21.15.93:0/1958718471 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2d78003cd0 con 0x7f2d801006c0 2023-12-14T18:24:35.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.306+0000 7f2d7dffb700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d700a11d0 0x7f2d700a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:35.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.307+0000 7f2d84c25700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d700a11d0 0x7f2d700a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:35.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.307+0000 7f2d7dffb700 1 -- 172.21.15.93:0/1958718471 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 7497+0+0 (secure 0 0 0) 0x7f2d780cee30 con 0x7f2d801006c0 2023-12-14T18:24:35.310 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.307+0000 7f2d84c25700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d700a11d0 0x7f2d700a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2d80068d00 tx=0x7f2d7400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:35.311 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.310+0000 7f2d7dffb700 1 -- 172.21.15.93:0/1958718471 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2d78098810 con 0x7f2d801006c0 2023-12-14T18:24:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3489910721' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-14T18:24:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-14T18:24:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:35.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:35.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.502 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:35.501+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) v1 -- 0x7f2d800fefa0 con 0x7f2d801006c0 2023-12-14T18:24:35.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3489910721' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-14T18:24:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-14T18:24:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:35.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:35.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3489910721' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:24:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:36.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.158+0000 7f2d7dffb700 1 -- 172.21.15.93:0/1958718471 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v78) v1 ==== 171+0+0 (secure 0 0 0) 0x7f2d7809be30 con 0x7f2d801006c0 2023-12-14T18:24:36.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.170+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d700a11d0 msgr2=0x7f2d700a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.170+0000 7f2d86e89700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d700a11d0 0x7f2d700a3680 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f2d80068d00 tx=0x7f2d7400b410 comp rx=0 tx=0).stop 2023-12-14T18:24:36.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.170+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d801006c0 msgr2=0x7f2d8019c3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.170+0000 7f2d86e89700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d801006c0 0x7f2d8019c3f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f2d7800e5a0 tx=0x7f2d7800e8b0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.175+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 shutdown_connections 2023-12-14T18:24:36.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.175+0000 7f2d86e89700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d700a11d0 0x7f2d700a3680 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.175+0000 7f2d86e89700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d801006c0 0x7f2d8019c3f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.175+0000 7f2d86e89700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d800ff3f0 0x7f2d80194e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.175+0000 7f2d86e89700 1 --2- 172.21.15.93:0/1958718471 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d800ffd80 0x7f2d80195380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.175+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 >> 172.21.15.93:0/1958718471 conn(0x7f2d800f9eb0 msgr2=0x7f2d800fc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:36.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.176+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 shutdown_connections 2023-12-14T18:24:36.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.176+0000 7f2d86e89700 1 -- 172.21.15.93:0/1958718471 wait complete. 2023-12-14T18:24:36.179 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data' created 2023-12-14T18:24:36.191 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-14T18:24:36.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.316+0000 7fef21cd5700 1 -- 172.21.15.93:0/471262196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c105f40 msgr2=0x7fef1c108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.316+0000 7fef21cd5700 1 --2- 172.21.15.93:0/471262196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c105f40 0x7fef1c108340 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fef1400b0a0 tx=0x7fef1400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.316+0000 7fef21cd5700 1 -- 172.21.15.93:0/471262196 shutdown_connections 2023-12-14T18:24:36.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.316+0000 7fef21cd5700 1 --2- 172.21.15.93:0/471262196 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c105f40 0x7fef1c108340 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.316+0000 7fef21cd5700 1 --2- 172.21.15.93:0/471262196 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef1c100d00 0x7fef1c1030e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.316+0000 7fef21cd5700 1 --2- 172.21.15.93:0/471262196 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c103620 0x7fef1c105a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.316+0000 7fef21cd5700 1 -- 172.21.15.93:0/471262196 >> 172.21.15.93:0/471262196 conn(0x7fef1c0fa740 msgr2=0x7fef1c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:36.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.316+0000 7fef21cd5700 1 -- 172.21.15.93:0/471262196 shutdown_connections 2023-12-14T18:24:36.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.316+0000 7fef21cd5700 1 -- 172.21.15.93:0/471262196 wait complete. 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef21cd5700 1 Processor -- start 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef21cd5700 1 -- start start 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef21cd5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c100d00 0x7fef1c100670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef21cd5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c103620 0x7fef1c0fecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef21cd5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef1c105f40 0x7fef1c0ff1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef21cd5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0ff990 con 0x7fef1c103620 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef21cd5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c0ffb00 con 0x7fef1c100d00 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef21cd5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fef1c1a00e0 con 0x7fef1c105f40 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c103620 0x7fef1c0fecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c103620 0x7fef1c0fecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56446/0 (socket says 172.21.15.93:56446) 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1affd700 1 -- 172.21.15.93:0/4156901826 learned_addr learned my addr 172.21.15.93:0/4156901826 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1b7fe700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c100d00 0x7fef1c100670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1bfff700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef1c105f40 0x7fef1c0ff1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:36.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1affd700 1 -- 172.21.15.93:0/4156901826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef1c105f40 msgr2=0x7fef1c0ff1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1affd700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef1c105f40 0x7fef1c0ff1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1affd700 1 -- 172.21.15.93:0/4156901826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c100d00 msgr2=0x7fef1c100670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1affd700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c100d00 0x7fef1c100670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.317+0000 7fef1affd700 1 -- 172.21.15.93:0/4156901826 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fef14009d00 con 0x7fef1c103620 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.318+0000 7fef1affd700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c103620 0x7fef1c0fecb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00d950 tx=0x7fef0c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.318+0000 7fef18ff9700 1 -- 172.21.15.93:0/4156901826 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c009940 con 0x7fef1c103620 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.318+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fef1c10bb80 con 0x7fef1c103620 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.318+0000 7fef18ff9700 1 -- 172.21.15.93:0/4156901826 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fef0c010460 con 0x7fef1c103620 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.318+0000 7fef18ff9700 1 -- 172.21.15.93:0/4156901826 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fef0c00f620 con 0x7fef1c103620 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.318+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fef1c10b9f0 con 0x7fef1c103620 2023-12-14T18:24:36.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.319+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fef1c0fc2b0 con 0x7fef1c103620 2023-12-14T18:24:36.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.320+0000 7fef18ff9700 1 -- 172.21.15.93:0/4156901826 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fef0c0108d0 con 0x7fef1c103620 2023-12-14T18:24:36.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.320+0000 7fef18ff9700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fef040a11d0 0x7fef040a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:36.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.320+0000 7fef18ff9700 1 -- 172.21.15.93:0/4156901826 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 7879+0+0 (secure 0 0 0) 0x7fef0c0cc620 con 0x7fef1c103620 2023-12-14T18:24:36.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.320+0000 7fef1b7fe700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fef040a11d0 0x7fef040a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:36.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.321+0000 7fef1b7fe700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fef040a11d0 0x7fef040a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fef10007c60 tx=0x7fef100073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:36.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.325+0000 7fef18ff9700 1 -- 172.21.15.93:0/4156901826 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fef0c095e80 con 0x7fef1c103620 2023-12-14T18:24:36.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:36 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:24:36.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:36 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:24:36.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:36 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:24:36.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:36 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:24:36.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:36 smithi160 ceph-mon[43840]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-14T18:24:36.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:36 smithi160 ceph-mon[43840]: osdmap e77: 12 total, 12 up, 12 in 2023-12-14T18:24:36.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:36 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1958718471' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-14T18:24:36.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:36.557 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.556+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fef1c061930 con 0x7fef1c103620 2023-12-14T18:24:36.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:36 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:24:36.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:36 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:24:36.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:36 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:24:36.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:36 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:24:36.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:36 smithi093 ceph-mon[42070]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-14T18:24:36.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:36 smithi093 ceph-mon[42070]: osdmap e77: 12 total, 12 up, 12 in 2023-12-14T18:24:36.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:36 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1958718471' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-14T18:24:36.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:36.567 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.567+0000 7fef18ff9700 1 -- 172.21.15.93:0/4156901826 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) v1 ==== 172+0+0 (secure 0 0 0) 0x7fef0c0b9020 con 0x7fef1c103620 2023-12-14T18:24:36.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.571+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fef040a11d0 msgr2=0x7fef040a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.571+0000 7fef21cd5700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fef040a11d0 0x7fef040a3680 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fef10007c60 tx=0x7fef100073d0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.571+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c103620 msgr2=0x7fef1c0fecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.571+0000 7fef21cd5700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c103620 0x7fef1c0fecb0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fef0c00d950 tx=0x7fef0c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:24:36.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.572+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 shutdown_connections 2023-12-14T18:24:36.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.572+0000 7fef21cd5700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fef040a11d0 0x7fef040a3680 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.572+0000 7fef21cd5700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fef1c105f40 0x7fef1c0ff1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.572+0000 7fef21cd5700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fef1c100d00 0x7fef1c100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.572+0000 7fef21cd5700 1 --2- 172.21.15.93:0/4156901826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fef1c103620 0x7fef1c0fecb0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.572+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 >> 172.21.15.93:0/4156901826 conn(0x7fef1c0fa740 msgr2=0x7fef1c101960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:36.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.572+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 shutdown_connections 2023-12-14T18:24:36.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.573+0000 7fef21cd5700 1 -- 172.21.15.93:0/4156901826 wait complete. 2023-12-14T18:24:36.586 INFO:teuthology.orchestra.run.smithi093.stderr:new fs with metadata pool 2 and data pool 3 2023-12-14T18:24:36.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:36 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:24:36.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:36 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:24:36.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:36 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:24:36.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:36 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:24:36.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:36 smithi097 ceph-mon[43486]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2023-12-14T18:24:36.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:36 smithi097 ceph-mon[43486]: osdmap e77: 12 total, 12 up, 12 in 2023-12-14T18:24:36.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:36 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1958718471' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2023-12-14T18:24:36.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:24:36.597 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2023-12-14T18:24:36.597 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 -- 172.21.15.93:0/4279398419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8103660 msgr2=0x7f26b8105a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 --2- 172.21.15.93:0/4279398419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8103660 0x7f26b8105a40 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f26a8009990 tx=0x7f26a8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 -- 172.21.15.93:0/4279398419 shutdown_connections 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 --2- 172.21.15.93:0/4279398419 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b8105f80 0x7f26b8108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 --2- 172.21.15.93:0/4279398419 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26b8100d40 0x7f26b8103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 --2- 172.21.15.93:0/4279398419 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8103660 0x7f26b8105a40 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 -- 172.21.15.93:0/4279398419 >> 172.21.15.93:0/4279398419 conn(0x7f26b80fa740 msgr2=0x7f26b80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 -- 172.21.15.93:0/4279398419 shutdown_connections 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 -- 172.21.15.93:0/4279398419 wait complete. 2023-12-14T18:24:36.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.723+0000 7f26be0e6700 1 Processor -- start 2023-12-14T18:24:36.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26be0e6700 1 -- start start 2023-12-14T18:24:36.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26be0e6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8100d40 0x7f26b81006b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:36.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26be0e6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b8103660 0x7f26b80fecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26be0e6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26b8105f80 0x7f26b80ff230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26be0e6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80ff9d0 con 0x7f26b8100d40 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26be0e6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b80ffb40 con 0x7f26b8103660 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26be0e6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26b81a0010 con 0x7f26b8105f80 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8100d40 0x7f26b81006b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8100d40 0x7f26b81006b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56464/0 (socket says 172.21.15.93:56464) 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b77fe700 1 -- 172.21.15.93:0/2361642859 learned_addr learned my addr 172.21.15.93:0/2361642859 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b7fff700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26b8105f80 0x7f26b80ff230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b6ffd700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b8103660 0x7f26b80fecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b77fe700 1 -- 172.21.15.93:0/2361642859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26b8105f80 msgr2=0x7f26b80ff230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b77fe700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26b8105f80 0x7f26b80ff230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b77fe700 1 -- 172.21.15.93:0/2361642859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b8103660 msgr2=0x7f26b80fecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b77fe700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b8103660 0x7f26b80fecf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:36.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b77fe700 1 -- 172.21.15.93:0/2361642859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26a80095f0 con 0x7f26b8100d40 2023-12-14T18:24:36.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b77fe700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8100d40 0x7f26b81006b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00d950 tx=0x7f26ac00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:36.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b4ff9700 1 -- 172.21.15.93:0/2361642859 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00f890 con 0x7f26b8100d40 2023-12-14T18:24:36.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26b810bc30 con 0x7f26b8100d40 2023-12-14T18:24:36.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b4ff9700 1 -- 172.21.15.93:0/2361642859 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f26ac010460 con 0x7f26b8100d40 2023-12-14T18:24:36.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.724+0000 7f26b4ff9700 1 -- 172.21.15.93:0/2361642859 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26ac00e610 con 0x7f26b8100d40 2023-12-14T18:24:36.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.725+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26b810baa0 con 0x7f26b8100d40 2023-12-14T18:24:36.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.725+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26b81a0750 con 0x7f26b8100d40 2023-12-14T18:24:36.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.726+0000 7f26b4ff9700 1 -- 172.21.15.93:0/2361642859 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f26ac0108d0 con 0x7f26b8100d40 2023-12-14T18:24:36.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.727+0000 7f26b4ff9700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f26a00a1150 0x7f26a00a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:36.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.727+0000 7f26b6ffd700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f26a00a1150 0x7f26a00a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:36.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.727+0000 7f26b4ff9700 1 -- 172.21.15.93:0/2361642859 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 7979+0+0 (secure 0 0 0) 0x7f26ac0cc580 con 0x7f26b8100d40 2023-12-14T18:24:36.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.728+0000 7f26b6ffd700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f26a00a1150 0x7f26a00a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26a800b5c0 tx=0x7f26a8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:36.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.731+0000 7f26b4ff9700 1 -- 172.21.15.93:0/2361642859 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f26ac095e20 con 0x7f26b8100d40 2023-12-14T18:24:36.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:36.929+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) v1 -- 0x7f26b81a0d00 con 0x7f26b8100d40 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: pgmap v217: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1958718471' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: osdmap e78: 12 total, 12 up, 12 in 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4156901826' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: osdmap e79: 12 total, 12 up, 12 in 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4156901826' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-14T18:24:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:37 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2361642859' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-14T18:24:37.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: pgmap v217: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1958718471' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-14T18:24:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: osdmap e78: 12 total, 12 up, 12 in 2023-12-14T18:24:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4156901826' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-14T18:24:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-14T18:24:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: osdmap e79: 12 total, 12 up, 12 in 2023-12-14T18:24:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4156901826' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-14T18:24:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-14T18:24:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-14T18:24:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:37 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2361642859' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-14T18:24:37.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.566+0000 7f26b4ff9700 1 -- 172.21.15.93:0/2361642859 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) v1 ==== 157+0+0 (secure 0 0 0) 0x7f26ac0160e0 con 0x7f26b8100d40 2023-12-14T18:24:37.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f26a00a1150 msgr2=0x7f26a00a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:37.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f26a00a1150 0x7f26a00a3600 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f26a800b5c0 tx=0x7f26a8005fb0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8100d40 msgr2=0x7f26b81006b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:37.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8100d40 0x7f26b81006b0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f26ac00d950 tx=0x7f26ac00dc60 comp rx=0 tx=0).stop 2023-12-14T18:24:37.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 shutdown_connections 2023-12-14T18:24:37.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f26a00a1150 0x7f26a00a3600 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f26b8100d40 0x7f26b81006b0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f26b8105f80 0x7f26b80ff230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 --2- 172.21.15.93:0/2361642859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f26b8103660 0x7f26b80fecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 >> 172.21.15.93:0/2361642859 conn(0x7f26b80fa740 msgr2=0x7f26b8102180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:37.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 shutdown_connections 2023-12-14T18:24:37.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.569+0000 7f26be0e6700 1 -- 172.21.15.93:0/2361642859 wait complete. 2023-12-14T18:24:37.585 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2023-12-14T18:24:37.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: pgmap v217: 33 pgs: 9 creating+peering, 1 active+clean, 23 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:24:37.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1958718471' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2023-12-14T18:24:37.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: osdmap e78: 12 total, 12 up, 12 in 2023-12-14T18:24:37.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4156901826' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2023-12-14T18:24:37.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: daemon mds.b assigned to filesystem cephfs as rank 0 2023-12-14T18:24:37.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: osdmap e79: 12 total, 12 up, 12 in 2023-12-14T18:24:37.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4156901826' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2023-12-14T18:24:37.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2023-12-14T18:24:37.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: daemon mds.b is now active in filesystem cephfs as rank 0 2023-12-14T18:24:37.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:37 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2361642859' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2023-12-14T18:24:37.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 -- 172.21.15.93:0/2653085380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 msgr2=0x7f826c1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:37.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 --2- 172.21.15.93:0/2653085380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 0x7f826c1087c0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f826400b0a0 tx=0x7f826400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 -- 172.21.15.93:0/2653085380 shutdown_connections 2023-12-14T18:24:37.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 --2- 172.21.15.93:0/2653085380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 0x7f826c1087c0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 --2- 172.21.15.93:0/2653085380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f826c0fe500 0x7f826c0fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 --2- 172.21.15.93:0/2653085380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f826c101570 0x7f826c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 -- 172.21.15.93:0/2653085380 >> 172.21.15.93:0/2653085380 conn(0x7f826c0f9ee0 msgr2=0x7f826c0fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:37.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 -- 172.21.15.93:0/2653085380 shutdown_connections 2023-12-14T18:24:37.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 -- 172.21.15.93:0/2653085380 wait complete. 2023-12-14T18:24:37.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 Processor -- start 2023-12-14T18:24:37.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 -- start start 2023-12-14T18:24:37.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f826c0fe500 0x7f826c100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:37.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f826c101570 0x7f826c0ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:37.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 0x7f826c0ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:37.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c100200 con 0x7f826c101f00 2023-12-14T18:24:37.724 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c100370 con 0x7f826c0fe500 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.719+0000 7f8273d64700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f826c1a00e0 con 0x7f826c101570 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8272301700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 0x7f826c0ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8272301700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 0x7f826c0ffa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56484/0 (socket says 172.21.15.93:56484) 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8272301700 1 -- 172.21.15.93:0/116336975 learned_addr learned my addr 172.21.15.93:0/116336975 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8271b00700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f826c0fe500 0x7f826c100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f82712ff700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f826c101570 0x7f826c0ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8272301700 1 -- 172.21.15.93:0/116336975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f826c101570 msgr2=0x7f826c0ff520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8272301700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f826c101570 0x7f826c0ff520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8272301700 1 -- 172.21.15.93:0/116336975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f826c0fe500 msgr2=0x7f826c100ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8272301700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f826c0fe500 0x7f826c100ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8272301700 1 -- 172.21.15.93:0/116336975 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8264009d00 con 0x7f826c101f00 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8272301700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 0x7f826c0ffa60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f82640062a0 tx=0x7f8264004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8262ffd700 1 -- 172.21.15.93:0/116336975 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826400e040 con 0x7f826c101f00 2023-12-14T18:24:37.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8262ffd700 1 -- 172.21.15.93:0/116336975 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f826400cec0 con 0x7f826c101f00 2023-12-14T18:24:37.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f826c10bb80 con 0x7f826c101f00 2023-12-14T18:24:37.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8262ffd700 1 -- 172.21.15.93:0/116336975 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f826401baf0 con 0x7f826c101f00 2023-12-14T18:24:37.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.720+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f826c10b9f0 con 0x7f826c101f00 2023-12-14T18:24:37.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.721+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f826c107260 con 0x7f826c101f00 2023-12-14T18:24:37.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.722+0000 7f8262ffd700 1 -- 172.21.15.93:0/116336975 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f826401bc50 con 0x7f826c101f00 2023-12-14T18:24:37.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.723+0000 7f8262ffd700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f82580a11d0 0x7f82580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:37.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.723+0000 7f8271b00700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f82580a11d0 0x7f82580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:37.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.723+0000 7f8262ffd700 1 -- 172.21.15.93:0/116336975 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 8205+0+0 (secure 0 0 0) 0x7f82640ce420 con 0x7f826c101f00 2023-12-14T18:24:37.727 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.723+0000 7f8271b00700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f82580a11d0 0x7f82580a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f826c067b10 tx=0x7f8268007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:37.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.727+0000 7f8262ffd700 1 -- 172.21.15.93:0/116336975 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8264097bc0 con 0x7f826c101f00 2023-12-14T18:24:37.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:37.917+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f826c061940 con 0x7f826c101f00 2023-12-14T18:24:38.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:38 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] up:active 2023-12-14T18:24:38.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:38 smithi093 ceph-mon[42070]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:24:38.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:38 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2361642859' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-14T18:24:38.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:38 smithi093 ceph-mon[42070]: osdmap e80: 12 total, 12 up, 12 in 2023-12-14T18:24:38.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:38 smithi093 ceph-mon[42070]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:24:38.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:38 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/116336975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-14T18:24:38.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:38 smithi093 ceph-mon[42070]: pgmap v221: 97 pgs: 23 creating+peering, 31 active+clean, 43 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-12-14T18:24:38.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:38 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] up:active 2023-12-14T18:24:38.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:38 smithi097 ceph-mon[43486]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:24:38.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:38 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2361642859' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-14T18:24:38.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:38 smithi097 ceph-mon[43486]: osdmap e80: 12 total, 12 up, 12 in 2023-12-14T18:24:38.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:38 smithi097 ceph-mon[43486]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:24:38.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:38 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/116336975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-14T18:24:38.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:38 smithi097 ceph-mon[43486]: pgmap v221: 97 pgs: 23 creating+peering, 31 active+clean, 43 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-12-14T18:24:38.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:38 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] up:active 2023-12-14T18:24:38.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:38 smithi160 ceph-mon[43840]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:24:38.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:38 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2361642859' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2023-12-14T18:24:38.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:38 smithi160 ceph-mon[43840]: osdmap e80: 12 total, 12 up, 12 in 2023-12-14T18:24:38.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:38 smithi160 ceph-mon[43840]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:24:38.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:38 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/116336975' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2023-12-14T18:24:38.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:38 smithi160 ceph-mon[43840]: pgmap v221: 97 pgs: 23 creating+peering, 31 active+clean, 43 unknown; 577 KiB data, 73 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2023-12-14T18:24:39.579 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.577+0000 7f8262ffd700 1 -- 172.21.15.93:0/116336975 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) v1 ==== 228+0+0 (secure 0 0 0) 0x7f826409b1e0 con 0x7f826c101f00 2023-12-14T18:24:39.583 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.582+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f82580a11d0 msgr2=0x7f82580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:39.583 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.582+0000 7f8273d64700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f82580a11d0 0x7f82580a3680 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f826c067b10 tx=0x7f8268007400 comp rx=0 tx=0).stop 2023-12-14T18:24:39.583 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.582+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 msgr2=0x7f826c0ffa60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:39.583 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.582+0000 7f8273d64700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 0x7f826c0ffa60 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f82640062a0 tx=0x7f8264004d00 comp rx=0 tx=0).stop 2023-12-14T18:24:39.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.586+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 shutdown_connections 2023-12-14T18:24:39.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.586+0000 7f8273d64700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f82580a11d0 0x7f82580a3680 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.586+0000 7f8273d64700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f826c101f00 0x7f826c0ffa60 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.586+0000 7f8273d64700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f826c0fe500 0x7f826c100ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.586+0000 7f8273d64700 1 --2- 172.21.15.93:0/116336975 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f826c101570 0x7f826c0ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.586+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 >> 172.21.15.93:0/116336975 conn(0x7f826c0f9ee0 msgr2=0x7f826c0fc0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:39.588 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.586+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 shutdown_connections 2023-12-14T18:24:39.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.587+0000 7f8273d64700 1 -- 172.21.15.93:0/116336975 wait complete. 2023-12-14T18:24:39.589 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data_ec' created 2023-12-14T18:24:39.601 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/277623274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101ef0 msgr2=0x7f2b381087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/277623274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101ef0 0x7f2b381087b0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f2b3000b0a0 tx=0x7f2b3000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/277623274 shutdown_connections 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/277623274 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101ef0 0x7f2b381087b0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/277623274 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b380fe4f0 0x7f2b380fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/277623274 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b38101560 0x7f2b381019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/277623274 >> 172.21.15.93:0/277623274 conn(0x7f2b380f9ed0 msgr2=0x7f2b380fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/277623274 shutdown_connections 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/277623274 wait complete. 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.727+0000 7f2b3e8d2700 1 Processor -- start 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b3e8d2700 1 -- start start 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b3e8d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b380fe4f0 0x7f2b381960a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:39.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b3e8d2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101560 0x7f2b381965e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b3e8d2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b38101ef0 0x7f2b3819d650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b3e8d2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3819dc20 con 0x7f2b38101560 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b3e8d2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3819dd60 con 0x7f2b38101ef0 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b3e8d2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b3819def0 con 0x7f2b380fe4f0 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b2ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101560 0x7f2b381965e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b2ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101560 0x7f2b381965e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56500/0 (socket says 172.21.15.93:56500) 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b2ffff700 1 -- 172.21.15.93:0/1907823076 learned_addr learned my addr 172.21.15.93:0/1907823076 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b37fff700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b380fe4f0 0x7f2b381960a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b2ffff700 1 -- 172.21.15.93:0/1907823076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b380fe4f0 msgr2=0x7f2b381960a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b2ffff700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b380fe4f0 0x7f2b381960a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b2ffff700 1 -- 172.21.15.93:0/1907823076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b38101ef0 msgr2=0x7f2b3819d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b2ffff700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b38101ef0 0x7f2b3819d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:39.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b2ffff700 1 -- 172.21.15.93:0/1907823076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b30009d00 con 0x7f2b38101560 2023-12-14T18:24:39.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b2ffff700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101560 0x7f2b381965e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000e9a0 tx=0x7f2b2000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:39.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b35ffb700 1 -- 172.21.15.93:0/1907823076 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20009790 con 0x7f2b38101560 2023-12-14T18:24:39.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b3810bb70 con 0x7f2b38101560 2023-12-14T18:24:39.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b35ffb700 1 -- 172.21.15.93:0/1907823076 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b2000cbd0 con 0x7f2b38101560 2023-12-14T18:24:39.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b35ffb700 1 -- 172.21.15.93:0/1907823076 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20018680 con 0x7f2b38101560 2023-12-14T18:24:39.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.728+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b3810b9e0 con 0x7f2b38101560 2023-12-14T18:24:39.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.729+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b3819e5e0 con 0x7f2b38101560 2023-12-14T18:24:39.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.730+0000 7f2b35ffb700 1 -- 172.21.15.93:0/1907823076 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2b2000cd40 con 0x7f2b38101560 2023-12-14T18:24:39.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.731+0000 7f2b35ffb700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b240a1150 0x7f2b240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:39.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.731+0000 7f2b37fff700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b240a1150 0x7f2b240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:39.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.731+0000 7f2b35ffb700 1 -- 172.21.15.93:0/1907823076 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 8698+0+0 (secure 0 0 0) 0x7f2b20014070 con 0x7f2b38101560 2023-12-14T18:24:39.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.731+0000 7f2b37fff700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b240a1150 0x7f2b240a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2b38067b10 tx=0x7f2b28009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:39.736 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.735+0000 7f2b35ffb700 1 -- 172.21.15.93:0/1907823076 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2b20096590 con 0x7f2b38101560 2023-12-14T18:24:39.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:39 smithi097 ceph-mon[43486]: osdmap e81: 12 total, 12 up, 12 in 2023-12-14T18:24:39.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:39.922+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) v1 -- 0x7f2b3819ee20 con 0x7f2b38101560 2023-12-14T18:24:39.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:39 smithi160 ceph-mon[43840]: osdmap e81: 12 total, 12 up, 12 in 2023-12-14T18:24:40.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:39 smithi093 ceph-mon[42070]: osdmap e81: 12 total, 12 up, 12 in 2023-12-14T18:24:40.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.584+0000 7f2b35ffb700 1 -- 172.21.15.93:0/1907823076 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v83) v1 ==== 169+0+0 (secure 0 0 0) 0x7f2b20094e60 con 0x7f2b38101560 2023-12-14T18:24:40.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.591+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b240a1150 msgr2=0x7f2b240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:40.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.591+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b240a1150 0x7f2b240a3600 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f2b38067b10 tx=0x7f2b28009260 comp rx=0 tx=0).stop 2023-12-14T18:24:40.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.591+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101560 msgr2=0x7f2b381965e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:40.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.591+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101560 0x7f2b381965e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000e9a0 tx=0x7f2b2000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.592+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 shutdown_connections 2023-12-14T18:24:40.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.592+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b240a1150 0x7f2b240a3600 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.592+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b38101ef0 0x7f2b3819d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.592+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b380fe4f0 0x7f2b381960a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.592+0000 7f2b3e8d2700 1 --2- 172.21.15.93:0/1907823076 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b38101560 0x7f2b381965e0 secure :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f2b2000e9a0 tx=0x7f2b2000ecb0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.592+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 >> 172.21.15.93:0/1907823076 conn(0x7f2b380f9ed0 msgr2=0x7f2b380fc130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:40.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.597+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 shutdown_connections 2023-12-14T18:24:40.604 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.602+0000 7f2b3e8d2700 1 -- 172.21.15.93:0/1907823076 wait complete. 2023-12-14T18:24:40.608 INFO:teuthology.orchestra.run.smithi093.stderr:set pool 4 allow_ec_overwrites to true 2023-12-14T18:24:40.621 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2023-12-14T18:24:40.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.744+0000 7fcad35be700 1 -- 172.21.15.93:0/2926282647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc0fe4d0 msgr2=0x7fcacc0fe8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:40.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.744+0000 7fcad35be700 1 --2- 172.21.15.93:0/2926282647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc0fe4d0 0x7fcacc0fe8e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fcac8009990 tx=0x7fcac8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 -- 172.21.15.93:0/2926282647 shutdown_connections 2023-12-14T18:24:40.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 --2- 172.21.15.93:0/2926282647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc0fe4d0 0x7fcacc0fe8e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 --2- 172.21.15.93:0/2926282647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcacc101e30 0x7fcacc1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 --2- 172.21.15.93:0/2926282647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcacc1014a0 0x7fcacc1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 -- 172.21.15.93:0/2926282647 >> 172.21.15.93:0/2926282647 conn(0x7fcacc0f9ed0 msgr2=0x7fcacc0fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:40.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 -- 172.21.15.93:0/2926282647 shutdown_connections 2023-12-14T18:24:40.748 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 -- 172.21.15.93:0/2926282647 wait complete. 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 Processor -- start 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 -- start start 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcacc0fe4d0 0x7fcacc195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcacc1014a0 0x7fcacc196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc101e30 0x7fcacc19d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc19db40 con 0x7fcacc101e30 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc19dc80 con 0x7fcacc1014a0 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.745+0000 7fcad35be700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcacc19de10 con 0x7fcacc0fe4d0 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad1b5b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc101e30 0x7fcacc19d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad1b5b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc101e30 0x7fcacc19d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56522/0 (socket says 172.21.15.93:56522) 2023-12-14T18:24:40.749 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad1b5b700 1 -- 172.21.15.93:0/591260456 learned_addr learned my addr 172.21.15.93:0/591260456 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:40.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad135a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcacc0fe4d0 0x7fcacc195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:40.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad0b59700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcacc1014a0 0x7fcacc196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:40.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad1b5b700 1 -- 172.21.15.93:0/591260456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcacc0fe4d0 msgr2=0x7fcacc195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:40.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad1b5b700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcacc0fe4d0 0x7fcacc195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad1b5b700 1 -- 172.21.15.93:0/591260456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcacc1014a0 msgr2=0x7fcacc196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:40.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad1b5b700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcacc1014a0 0x7fcacc196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:40.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad1b5b700 1 -- 172.21.15.93:0/591260456 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcac80095f0 con 0x7fcacc101e30 2023-12-14T18:24:40.750 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.746+0000 7fcad1b5b700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc101e30 0x7fcacc19d570 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcac4007d70 tx=0x7fcac400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:40.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.747+0000 7fcabe7fc700 1 -- 172.21.15.93:0/591260456 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac400ac90 con 0x7fcacc101e30 2023-12-14T18:24:40.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.748+0000 7fcabe7fc700 1 -- 172.21.15.93:0/591260456 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcac400adf0 con 0x7fcacc101e30 2023-12-14T18:24:40.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.748+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcacc10bbf0 con 0x7fcacc101e30 2023-12-14T18:24:40.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.748+0000 7fcabe7fc700 1 -- 172.21.15.93:0/591260456 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcac40115e0 con 0x7fcacc101e30 2023-12-14T18:24:40.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.748+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcacc10ba60 con 0x7fcacc101e30 2023-12-14T18:24:40.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.748+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcacc107230 con 0x7fcacc101e30 2023-12-14T18:24:40.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.749+0000 7fcabe7fc700 1 -- 172.21.15.93:0/591260456 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcac4012420 con 0x7fcacc101e30 2023-12-14T18:24:40.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.750+0000 7fcabe7fc700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcab80a1250 0x7fcab80a3700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:40.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.750+0000 7fcad135a700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcab80a1250 0x7fcab80a3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:40.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.750+0000 7fcabe7fc700 1 -- 172.21.15.93:0/591260456 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 8698+0+0 (secure 0 0 0) 0x7fcac40cccb0 con 0x7fcacc101e30 2023-12-14T18:24:40.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.750+0000 7fcad135a700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcab80a1250 0x7fcab80a3700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcac8006010 tx=0x7fcac8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:40.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.754+0000 7fcabe7fc700 1 -- 172.21.15.93:0/591260456 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcac40961d0 con 0x7fcacc101e30 2023-12-14T18:24:40.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:40 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/116336975' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-14T18:24:40.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:40 smithi097 ceph-mon[43486]: osdmap e82: 12 total, 12 up, 12 in 2023-12-14T18:24:40.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:40 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1907823076' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-14T18:24:40.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:40 smithi097 ceph-mon[43486]: pgmap v224: 129 pgs: 25 creating+peering, 58 active+clean, 46 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 3 op/s 2023-12-14T18:24:40.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.993+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) v1 -- 0x7fcacc02cfe0 con 0x7fcacc101e30 2023-12-14T18:24:40.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:40 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/116336975' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-14T18:24:40.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:40 smithi160 ceph-mon[43840]: osdmap e82: 12 total, 12 up, 12 in 2023-12-14T18:24:40.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:40 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1907823076' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-14T18:24:40.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:40 smithi160 ceph-mon[43840]: pgmap v224: 129 pgs: 25 creating+peering, 58 active+clean, 46 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 3 op/s 2023-12-14T18:24:41.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:40.999+0000 7fcabe7fc700 1 -- 172.21.15.93:0/591260456 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) v1 ==== 137+0+0 (secure 0 0 0) 0x7fcac40997f0 con 0x7fcacc101e30 2023-12-14T18:24:41.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcab80a1250 msgr2=0x7fcab80a3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcab80a1250 0x7fcab80a3700 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fcac8006010 tx=0x7fcac8005960 comp rx=0 tx=0).stop 2023-12-14T18:24:41.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc101e30 msgr2=0x7fcacc19d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc101e30 0x7fcacc19d570 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcac4007d70 tx=0x7fcac400a340 comp rx=0 tx=0).stop 2023-12-14T18:24:41.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 shutdown_connections 2023-12-14T18:24:41.003 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcab80a1250 0x7fcab80a3700 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcacc101e30 0x7fcacc19d570 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcacc0fe4d0 0x7fcacc195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 --2- 172.21.15.93:0/591260456 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcacc1014a0 0x7fcacc196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 >> 172.21.15.93:0/591260456 conn(0x7fcacc0f9ed0 msgr2=0x7fcacc0fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:41.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 shutdown_connections 2023-12-14T18:24:41.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.001+0000 7fcad35be700 1 -- 172.21.15.93:0/591260456 wait complete. 2023-12-14T18:24:41.004 INFO:teuthology.orchestra.run.smithi093.stderr:added data pool 4 to fsmap 2023-12-14T18:24:41.014 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:40 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/116336975' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2023-12-14T18:24:41.014 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:40 smithi093 ceph-mon[42070]: osdmap e82: 12 total, 12 up, 12 in 2023-12-14T18:24:41.014 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:40 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1907823076' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2023-12-14T18:24:41.014 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:40 smithi093 ceph-mon[42070]: pgmap v224: 129 pgs: 25 creating+peering, 58 active+clean, 46 unknown; 577 KiB data, 74 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s wr, 3 op/s 2023-12-14T18:24:41.015 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:41.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.135+0000 7f1998925700 1 -- 172.21.15.93:0/354025632 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19940ff110 msgr2=0x7f19940ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.135+0000 7f1998925700 1 --2- 172.21.15.93:0/354025632 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19940ff110 0x7f19940ff520 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1984009990 tx=0x7f1984009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.135+0000 7f1998925700 1 -- 172.21.15.93:0/354025632 shutdown_connections 2023-12-14T18:24:41.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.135+0000 7f1998925700 1 --2- 172.21.15.93:0/354025632 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19940ff110 0x7f19940ff520 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.135+0000 7f1998925700 1 --2- 172.21.15.93:0/354025632 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1994100c90 0x7f1994107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.135+0000 7f1998925700 1 --2- 172.21.15.93:0/354025632 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1994100300 0x7f1994100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.135+0000 7f1998925700 1 -- 172.21.15.93:0/354025632 >> 172.21.15.93:0/354025632 conn(0x7f19940fa6f0 msgr2=0x7f19940fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:41.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.135+0000 7f1998925700 1 -- 172.21.15.93:0/354025632 shutdown_connections 2023-12-14T18:24:41.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.135+0000 7f1998925700 1 -- 172.21.15.93:0/354025632 wait complete. 2023-12-14T18:24:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.136+0000 7f1998925700 1 Processor -- start 2023-12-14T18:24:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.136+0000 7f1998925700 1 -- start start 2023-12-14T18:24:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1998925700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19940ff110 0x7f1994195030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1998925700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1994100300 0x7f1994195570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1998925700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994100c90 0x7f199419c5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1998925700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419cbb0 con 0x7f1994100c90 2023-12-14T18:24:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1998925700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419ccf0 con 0x7f19940ff110 2023-12-14T18:24:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1998925700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f199419ce80 con 0x7f1994100300 2023-12-14T18:24:41.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1992d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994100c90 0x7f199419c5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1992d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994100c90 0x7f199419c5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56538/0 (socket says 172.21.15.93:56538) 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1992d9d700 1 -- 172.21.15.93:0/2050060176 learned_addr learned my addr 172.21.15.93:0/2050060176 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f199259c700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19940ff110 0x7f1994195030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1991d9b700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1994100300 0x7f1994195570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1992d9d700 1 -- 172.21.15.93:0/2050060176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1994100300 msgr2=0x7f1994195570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1992d9d700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1994100300 0x7f1994195570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1992d9d700 1 -- 172.21.15.93:0/2050060176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19940ff110 msgr2=0x7f1994195030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1992d9d700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19940ff110 0x7f1994195030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1992d9d700 1 -- 172.21.15.93:0/2050060176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19840095f0 con 0x7f1994100c90 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1992d9d700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994100c90 0x7f199419c5e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f198c00b970 tx=0x7f198c00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:41.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f19837fe700 1 -- 172.21.15.93:0/2050060176 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c00c930 con 0x7f1994100c90 2023-12-14T18:24:41.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f199410ab70 con 0x7f1994100c90 2023-12-14T18:24:41.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f19837fe700 1 -- 172.21.15.93:0/2050060176 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f198c014440 con 0x7f1994100c90 2023-12-14T18:24:41.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f19837fe700 1 -- 172.21.15.93:0/2050060176 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f198c0075d0 con 0x7f1994100c90 2023-12-14T18:24:41.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.137+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f199410a9e0 con 0x7f1994100c90 2023-12-14T18:24:41.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.138+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1994106180 con 0x7f1994100c90 2023-12-14T18:24:41.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.140+0000 7f19837fe700 1 -- 172.21.15.93:0/2050060176 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f198c0145b0 con 0x7f1994100c90 2023-12-14T18:24:41.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.140+0000 7f19837fe700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f197c0a1150 0x7f197c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.140+0000 7f199259c700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f197c0a1150 0x7f197c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.140+0000 7f19837fe700 1 -- 172.21.15.93:0/2050060176 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f198c006e40 con 0x7f1994100c90 2023-12-14T18:24:41.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.141+0000 7f199259c700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f197c0a1150 0x7f197c0a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f198400b5c0 tx=0x7f1984005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:41.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.144+0000 7f19837fe700 1 -- 172.21.15.93:0/2050060176 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f198c095ba0 con 0x7f1994100c90 2023-12-14T18:24:41.377 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.376+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f199402cfe0 con 0x7f1994100c90 2023-12-14T18:24:41.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.376+0000 7f19837fe700 1 -- 172.21.15.93:0/2050060176 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10738 (secure 0 0 0) 0x7f198c0991c0 con 0x7f1994100c90 2023-12-14T18:24:41.378 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:41.378 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:24:40.995786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.378+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f197c0a1150 msgr2=0x7f197c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f197c0a1150 0x7f197c0a3600 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f198400b5c0 tx=0x7f1984005960 comp rx=0 tx=0).stop 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994100c90 msgr2=0x7f199419c5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994100c90 0x7f199419c5e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f198c00b970 tx=0x7f198c00bc80 comp rx=0 tx=0).stop 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 shutdown_connections 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f197c0a1150 0x7f197c0a3600 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1994100c90 0x7f199419c5e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19940ff110 0x7f1994195030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 --2- 172.21.15.93:0/2050060176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1994100300 0x7f1994195570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 >> 172.21.15.93:0/2050060176 conn(0x7f19940fa6f0 msgr2=0x7f19940fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:41.381 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 shutdown_connections 2023-12-14T18:24:41.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.379+0000 7f1998925700 1 -- 172.21.15.93:0/2050060176 wait complete. 2023-12-14T18:24:41.382 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 17 2023-12-14T18:24:41.393 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:24:41.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.512+0000 7fb02cf3f700 1 -- 172.21.15.93:0/1244974407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 msgr2=0x7fb028100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.512+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/1244974407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 0x7fb028100780 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb018009990 tx=0x7fb018009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.512+0000 7fb02cf3f700 1 -- 172.21.15.93:0/1244974407 shutdown_connections 2023-12-14T18:24:41.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.512+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/1244974407 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028100cc0 0x7fb028107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.512+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/1244974407 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0280ff140 0x7fb0280ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.512+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/1244974407 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 0x7fb028100780 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.512+0000 7fb02cf3f700 1 -- 172.21.15.93:0/1244974407 >> 172.21.15.93:0/1244974407 conn(0x7fb0280fa720 msgr2=0x7fb0280fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:41.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.512+0000 7fb02cf3f700 1 -- 172.21.15.93:0/1244974407 shutdown_connections 2023-12-14T18:24:41.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.512+0000 7fb02cf3f700 1 -- 172.21.15.93:0/1244974407 wait complete. 2023-12-14T18:24:41.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb02cf3f700 1 Processor -- start 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb02cf3f700 1 -- start start 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb02cf3f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0280ff140 0x7fb028194e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb02cf3f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 0x7fb0281953b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb02cf3f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028100cc0 0x7fb02819c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb02cf3f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02819c9f0 con 0x7fb028100330 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb02cf3f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02819cb30 con 0x7fb028100cc0 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb02cf3f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb02819ccc0 con 0x7fb0280ff140 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb025d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 0x7fb0281953b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb025d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 0x7fb0281953b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49202/0 (socket says 172.21.15.93:49202) 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb025d9b700 1 -- 172.21.15.93:0/890335528 learned_addr learned my addr 172.21.15.93:0/890335528 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb02659c700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0280ff140 0x7fb028194e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.517 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb025d9b700 1 -- 172.21.15.93:0/890335528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0280ff140 msgr2=0x7fb028194e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb025d9b700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0280ff140 0x7fb028194e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb025d9b700 1 -- 172.21.15.93:0/890335528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028100cc0 msgr2=0x7fb02819c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb026d9d700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028100cc0 0x7fb02819c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb025d9b700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028100cc0 0x7fb02819c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb025d9b700 1 -- 172.21.15.93:0/890335528 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0180095f0 con 0x7fb028100330 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.513+0000 7fb025d9b700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 0x7fb0281953b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb018005910 tx=0x7fb018004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.514+0000 7fb0177fe700 1 -- 172.21.15.93:0/890335528 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb01801d070 con 0x7fb028100330 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.514+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb02810ab00 con 0x7fb028100330 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.514+0000 7fb0177fe700 1 -- 172.21.15.93:0/890335528 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb018003f60 con 0x7fb028100330 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.514+0000 7fb0177fe700 1 -- 172.21.15.93:0/890335528 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb018022da0 con 0x7fb028100330 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.514+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb02810a970 con 0x7fb028100330 2023-12-14T18:24:41.518 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.515+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb028106070 con 0x7fb028100330 2023-12-14T18:24:41.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.516+0000 7fb0177fe700 1 -- 172.21.15.93:0/890335528 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb018003aa0 con 0x7fb028100330 2023-12-14T18:24:41.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.517+0000 7fb0177fe700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb0100a1150 0x7fb0100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.517+0000 7fb02659c700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb0100a1150 0x7fb0100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.517+0000 7fb0177fe700 1 -- 172.21.15.93:0/890335528 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb0180ce690 con 0x7fb028100330 2023-12-14T18:24:41.519 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.517+0000 7fb02659c700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb0100a1150 0x7fb0100a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb028100190 tx=0x7fb01c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:41.522 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.520+0000 7fb0177fe700 1 -- 172.21.15.93:0/890335528 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb018097c80 con 0x7fb028100330 2023-12-14T18:24:41.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.712+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb02802cfe0 con 0x7fb028100330 2023-12-14T18:24:41.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.714+0000 7fb0177fe700 1 -- 172.21.15.93:0/890335528 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7fb01809b2a0 con 0x7fb028100330 2023-12-14T18:24:41.716 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:41.716 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb0100a1150 msgr2=0x7fb0100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb0100a1150 0x7fb0100a3600 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fb028100190 tx=0x7fb01c00b410 comp rx=0 tx=0).stop 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 msgr2=0x7fb0281953b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 0x7fb0281953b0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fb018005910 tx=0x7fb018004730 comp rx=0 tx=0).stop 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 shutdown_connections 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb0100a1150 0x7fb0100a3600 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb028100cc0 0x7fb02819c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb0280ff140 0x7fb028194e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 --2- 172.21.15.93:0/890335528 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb028100330 0x7fb0281953b0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 >> 172.21.15.93:0/890335528 conn(0x7fb0280fa720 msgr2=0x7fb0280fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:41.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 shutdown_connections 2023-12-14T18:24:41.719 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.716+0000 7fb02cf3f700 1 -- 172.21.15.93:0/890335528 wait complete. 2023-12-14T18:24:41.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:41 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1907823076' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-14T18:24:41.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:41 smithi093 ceph-mon[42070]: osdmap e83: 12 total, 12 up, 12 in 2023-12-14T18:24:41.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:41 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/591260456' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-14T18:24:41.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:41 smithi093 ceph-mon[42070]: osdmap e84: 12 total, 12 up, 12 in 2023-12-14T18:24:41.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:41 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/591260456' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-14T18:24:41.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:41 smithi093 ceph-mon[42070]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:24:41.729 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:41 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2050060176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:41.730 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:24:41.842 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:24:41 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-b[68560]: 2023-12-14T18:24:41.752+0000 7f94e7277700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-14T18:24:41.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:41 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1907823076' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-14T18:24:41.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:41 smithi097 ceph-mon[43486]: osdmap e83: 12 total, 12 up, 12 in 2023-12-14T18:24:41.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:41 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/591260456' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-14T18:24:41.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:41 smithi097 ceph-mon[43486]: osdmap e84: 12 total, 12 up, 12 in 2023-12-14T18:24:41.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:41 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/591260456' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-14T18:24:41.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:41 smithi097 ceph-mon[43486]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:24:41.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:41 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2050060176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:41.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 -- 172.21.15.93:0/932230862 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100300 msgr2=0x7f3a00100750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 --2- 172.21.15.93:0/932230862 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100300 0x7f3a00100750 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f39f0009990 tx=0x7f39f0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 -- 172.21.15.93:0/932230862 shutdown_connections 2023-12-14T18:24:41.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 --2- 172.21.15.93:0/932230862 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a00100c90 0x7f3a00107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 --2- 172.21.15.93:0/932230862 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a000ff110 0x7f3a000ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 --2- 172.21.15.93:0/932230862 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100300 0x7f3a00100750 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 -- 172.21.15.93:0/932230862 >> 172.21.15.93:0/932230862 conn(0x7f3a000fa6f0 msgr2=0x7f3a000fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 -- 172.21.15.93:0/932230862 shutdown_connections 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 -- 172.21.15.93:0/932230862 wait complete. 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.850+0000 7f3a055d7700 1 Processor -- start 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f3a055d7700 1 -- start start 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f3a055d7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a000ff110 0x7f3a00195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f3a055d7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a00100300 0x7f3a00196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f3a055d7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100c90 0x7f3a0019d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f3a055d7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0019db70 con 0x7f3a00100c90 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f3a055d7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0019dcb0 con 0x7f3a00100300 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f3a055d7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a0019de40 con 0x7f3a000ff110 2023-12-14T18:24:41.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39ff7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100c90 0x7f3a0019d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39ff7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100c90 0x7f3a0019d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49216/0 (socket says 172.21.15.93:49216) 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39ff7fe700 1 -- 172.21.15.93:0/613343527 learned_addr learned my addr 172.21.15.93:0/613343527 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39fe7fc700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a00100300 0x7f3a00196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39feffd700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a000ff110 0x7f3a00195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39ff7fe700 1 -- 172.21.15.93:0/613343527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a000ff110 msgr2=0x7f3a00195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39ff7fe700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a000ff110 0x7f3a00195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39ff7fe700 1 -- 172.21.15.93:0/613343527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a00100300 msgr2=0x7f3a00196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39ff7fe700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a00100300 0x7f3a00196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39ff7fe700 1 -- 172.21.15.93:0/613343527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39f00095f0 con 0x7f3a00100c90 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39ff7fe700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100c90 0x7f3a0019d5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007ee0 tx=0x7f39f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.851+0000 7f39e7fff700 1 -- 172.21.15.93:0/613343527 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8017070 con 0x7f3a00100c90 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.852+0000 7f39e7fff700 1 -- 172.21.15.93:0/613343527 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f39f800f040 con 0x7f3a00100c90 2023-12-14T18:24:41.856 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.852+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a0010aa80 con 0x7f3a00100c90 2023-12-14T18:24:41.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.852+0000 7f39e7fff700 1 -- 172.21.15.93:0/613343527 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39f8005740 con 0x7f3a00100c90 2023-12-14T18:24:41.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.852+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a0010a8f0 con 0x7f3a00100c90 2023-12-14T18:24:41.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.852+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a0019e4e0 con 0x7f3a00100c90 2023-12-14T18:24:41.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.853+0000 7f39e7fff700 1 -- 172.21.15.93:0/613343527 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f39f800dc60 con 0x7f3a00100c90 2023-12-14T18:24:41.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.854+0000 7f39e7fff700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f39e80a1150 0x7f39e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:41.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.854+0000 7f39e7fff700 1 -- 172.21.15.93:0/613343527 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f39f80cc950 con 0x7f3a00100c90 2023-12-14T18:24:41.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.854+0000 7f39feffd700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f39e80a1150 0x7f39e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:41.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.855+0000 7f39feffd700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f39e80a1150 0x7f39e80a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39f4005950 tx=0x7f39f40058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:41.859 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:41.857+0000 7f39e7fff700 1 -- 172.21.15.93:0/613343527 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f39f8095eb0 con 0x7f3a00100c90 2023-12-14T18:24:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:41 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1907823076' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2023-12-14T18:24:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:41 smithi160 ceph-mon[43840]: osdmap e83: 12 total, 12 up, 12 in 2023-12-14T18:24:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:41 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/591260456' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2023-12-14T18:24:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:41 smithi160 ceph-mon[43840]: osdmap e84: 12 total, 12 up, 12 in 2023-12-14T18:24:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:41 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/591260456' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2023-12-14T18:24:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:41 smithi160 ceph-mon[43840]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:24:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:41 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2050060176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.052+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3a0019ea90 con 0x7f3a00100c90 2023-12-14T18:24:42.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.053+0000 7f39e7fff700 1 -- 172.21.15.93:0/613343527 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7f39f80994d0 con 0x7f3a00100c90 2023-12-14T18:24:42.055 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:42.055 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:42.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f39e80a1150 msgr2=0x7f39e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f39e80a1150 0x7f39e80a3600 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f39f4005950 tx=0x7f39f40058e0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100c90 msgr2=0x7f3a0019d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.057 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100c90 0x7f3a0019d5a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f39f8007ee0 tx=0x7f39f800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 shutdown_connections 2023-12-14T18:24:42.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f39e80a1150 0x7f39e80a3600 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3a00100c90 0x7f3a0019d5a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3a000ff110 0x7f3a00195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 --2- 172.21.15.93:0/613343527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3a00100300 0x7f3a00196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 >> 172.21.15.93:0/613343527 conn(0x7f3a000fa6f0 msgr2=0x7f3a00067e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 shutdown_connections 2023-12-14T18:24:42.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.055+0000 7f3a055d7700 1 -- 172.21.15.93:0/613343527 wait complete. 2023-12-14T18:24:42.082 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:24:42.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.201+0000 7f2d7bb00700 1 -- 172.21.15.93:0/1199417915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 msgr2=0x7f2d740ff7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.201+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/1199417915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 0x7f2d740ff7b0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009990 tx=0x7f2d64009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.201+0000 7f2d7bb00700 1 -- 172.21.15.93:0/1199417915 shutdown_connections 2023-12-14T18:24:42.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.201+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/1199417915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d740ffcf0 0x7f2d741075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.201+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/1199417915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d741006f0 0x7f2d74100b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.201+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/1199417915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 0x7f2d740ff7b0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.201+0000 7f2d7bb00700 1 -- 172.21.15.93:0/1199417915 >> 172.21.15.93:0/1199417915 conn(0x7f2d740f9e80 msgr2=0x7f2d740fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 -- 172.21.15.93:0/1199417915 shutdown_connections 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 -- 172.21.15.93:0/1199417915 wait complete. 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 Processor -- start 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 -- start start 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 0x7f2d740feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ffcf0 0x7f2d7419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d741006f0 0x7f2d7419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419fc10 con 0x7f2d740ff340 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419fd80 con 0x7f2d740ffcf0 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7bb00700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d7419ff40 con 0x7f2d741006f0 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7989c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 0x7f2d740feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7989c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 0x7f2d740feda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49226/0 (socket says 172.21.15.93:49226) 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.202+0000 7f2d7989c700 1 -- 172.21.15.93:0/3337225571 learned_addr learned my addr 172.21.15.93:0/3337225571 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:42.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7a09d700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d741006f0 0x7f2d7419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7909b700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ffcf0 0x7f2d7419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7989c700 1 -- 172.21.15.93:0/3337225571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d741006f0 msgr2=0x7f2d7419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7989c700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d741006f0 0x7f2d7419f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7989c700 1 -- 172.21.15.93:0/3337225571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ffcf0 msgr2=0x7f2d7419d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7989c700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ffcf0 0x7f2d7419d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7989c700 1 -- 172.21.15.93:0/3337225571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d640095f0 con 0x7f2d740ff340 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7909b700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ffcf0 0x7f2d7419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7989c700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 0x7f2d740feda0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000c990 tx=0x7f2d7000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d6affd700 1 -- 172.21.15.93:0/3337225571 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70007b00 con 0x7f2d740ff340 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d6affd700 1 -- 172.21.15.93:0/3337225571 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2d70007c60 con 0x7f2d740ff340 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d7410aa90 con 0x7f2d740ff340 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d6affd700 1 -- 172.21.15.93:0/3337225571 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d70018660 con 0x7f2d740ff340 2023-12-14T18:24:42.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.203+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d7410a900 con 0x7f2d740ff340 2023-12-14T18:24:42.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.204+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d741a0690 con 0x7f2d740ff340 2023-12-14T18:24:42.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.205+0000 7f2d6affd700 1 -- 172.21.15.93:0/3337225571 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2d70004020 con 0x7f2d740ff340 2023-12-14T18:24:42.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.206+0000 7f2d6affd700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d600a1150 0x7f2d600a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.206+0000 7f2d7909b700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d600a1150 0x7f2d600a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.206+0000 7f2d6affd700 1 -- 172.21.15.93:0/3337225571 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2d700ccc60 con 0x7f2d740ff340 2023-12-14T18:24:42.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.206+0000 7f2d7909b700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d600a1150 0x7f2d600a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b5c0 tx=0x7f2d64005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.209+0000 7f2d6affd700 1 -- 172.21.15.93:0/3337225571 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2d70096250 con 0x7f2d740ff340 2023-12-14T18:24:42.448 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.446+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2d74105fc0 con 0x7f2d740ff340 2023-12-14T18:24:42.449 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.446+0000 7f2d6affd700 1 -- 172.21.15.93:0/3337225571 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+10738 (secure 0 0 0) 0x7f2d70099870 con 0x7f2d740ff340 2023-12-14T18:24:42.449 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:42.449 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":7},{"gid":24422,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":6},{"gid":24424,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":8},{"gid":24440,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":9},{"gid":24442,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:24:40.995786+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":24461},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1},"id":1}]} 2023-12-14T18:24:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.448+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d600a1150 msgr2=0x7f2d600a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.448+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d600a1150 0x7f2d600a3600 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400b5c0 tx=0x7f2d64005fb0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.448+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 msgr2=0x7f2d740feda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.448+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 0x7f2d740feda0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f2d7000c990 tx=0x7f2d7000cd50 comp rx=0 tx=0).stop 2023-12-14T18:24:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.449+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 shutdown_connections 2023-12-14T18:24:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.449+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2d600a1150 0x7f2d600a3600 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.449+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2d740ff340 0x7f2d740feda0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.449+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2d741006f0 0x7f2d7419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.451 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.449+0000 7f2d7bb00700 1 --2- 172.21.15.93:0/3337225571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2d740ffcf0 0x7f2d7419d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.449+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 >> 172.21.15.93:0/3337225571 conn(0x7f2d740f9e80 msgr2=0x7f2d740fc0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.449+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 shutdown_connections 2023-12-14T18:24:42.452 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.449+0000 7f2d7bb00700 1 -- 172.21.15.93:0/3337225571 wait complete. 2023-12-14T18:24:42.452 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 17 2023-12-14T18:24:42.464 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:24:42.589 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.587+0000 7f84279b3700 1 -- 172.21.15.93:0/4265464105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100350 msgr2=0x7f84201007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.587+0000 7f84279b3700 1 --2- 172.21.15.93:0/4265464105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100350 0x7f84201007a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f8410009990 tx=0x7f8410009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.587+0000 7f84279b3700 1 -- 172.21.15.93:0/4265464105 shutdown_connections 2023-12-14T18:24:42.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.587+0000 7f84279b3700 1 --2- 172.21.15.93:0/4265464105 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8420100ce0 0x7f84201075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.587+0000 7f84279b3700 1 --2- 172.21.15.93:0/4265464105 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84200ff160 0x7f84200ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.587+0000 7f84279b3700 1 --2- 172.21.15.93:0/4265464105 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100350 0x7f84201007a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.587+0000 7f84279b3700 1 -- 172.21.15.93:0/4265464105 >> 172.21.15.93:0/4265464105 conn(0x7f84200fa780 msgr2=0x7f84200fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.587+0000 7f84279b3700 1 -- 172.21.15.93:0/4265464105 shutdown_connections 2023-12-14T18:24:42.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.587+0000 7f84279b3700 1 -- 172.21.15.93:0/4265464105 wait complete. 2023-12-14T18:24:42.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.588+0000 7f84279b3700 1 Processor -- start 2023-12-14T18:24:42.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.588+0000 7f84279b3700 1 -- start start 2023-12-14T18:24:42.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.588+0000 7f84279b3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84200ff160 0x7f8420069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.588+0000 7f84279b3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8420100350 0x7f842019acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.588+0000 7f84279b3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100ce0 0x7f842019d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.593 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.588+0000 7f84279b3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842019db10 con 0x7f8420100ce0 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.588+0000 7f84279b3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842019dc80 con 0x7f84200ff160 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.588+0000 7f84279b3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f842019de40 con 0x7f8420100350 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8425f50700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100ce0 0x7f842019d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8425f50700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100ce0 0x7f842019d340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49242/0 (socket says 172.21.15.93:49242) 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8425f50700 1 -- 172.21.15.93:0/2845483413 learned_addr learned my addr 172.21.15.93:0/2845483413 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f842574f700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84200ff160 0x7f8420069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8424f4e700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8420100350 0x7f842019acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8425f50700 1 -- 172.21.15.93:0/2845483413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8420100350 msgr2=0x7f842019acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8425f50700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8420100350 0x7f842019acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8425f50700 1 -- 172.21.15.93:0/2845483413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84200ff160 msgr2=0x7f8420069b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.594 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8425f50700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84200ff160 0x7f8420069b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8425f50700 1 -- 172.21.15.93:0/2845483413 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84100095f0 con 0x7f8420100ce0 2023-12-14T18:24:42.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f8425f50700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100ce0 0x7f842019d340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8418007b70 tx=0x7f8418007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f84167fc700 1 -- 172.21.15.93:0/2845483413 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8418003c50 con 0x7f8420100ce0 2023-12-14T18:24:42.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f84167fc700 1 -- 172.21.15.93:0/2845483413 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f841800f040 con 0x7f8420100ce0 2023-12-14T18:24:42.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f842010aac0 con 0x7f8420100ce0 2023-12-14T18:24:42.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f84167fc700 1 -- 172.21.15.93:0/2845483413 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84180044e0 con 0x7f8420100ce0 2023-12-14T18:24:42.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.589+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f842010a930 con 0x7f8420100ce0 2023-12-14T18:24:42.595 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.590+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84201066f0 con 0x7f8420100ce0 2023-12-14T18:24:42.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.593+0000 7f84167fc700 1 -- 172.21.15.93:0/2845483413 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8418004640 con 0x7f8420100ce0 2023-12-14T18:24:42.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.594+0000 7f84167fc700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f840c0a11d0 0x7f840c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.594+0000 7f84167fc700 1 -- 172.21.15.93:0/2845483413 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f84180cc6e0 con 0x7f8420100ce0 2023-12-14T18:24:42.596 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.594+0000 7f842574f700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f840c0a11d0 0x7f840c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.597 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.594+0000 7f842574f700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f840c0a11d0 0x7f840c0a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f84201001b0 tx=0x7f841c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.597+0000 7f84167fc700 1 -- 172.21.15.93:0/2845483413 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8418095c40 con 0x7f8420100ce0 2023-12-14T18:24:42.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.790+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f842002cfe0 con 0x7f8420100ce0 2023-12-14T18:24:42.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.792+0000 7f84167fc700 1 -- 172.21.15.93:0/2845483413 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7f8418023020 con 0x7f8420100ce0 2023-12-14T18:24:42.794 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:42.795 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:42.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f840c0a11d0 msgr2=0x7f840c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f840c0a11d0 0x7f840c0a3680 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f84201001b0 tx=0x7f841c009260 comp rx=0 tx=0).stop 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100ce0 msgr2=0x7f842019d340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100ce0 0x7f842019d340 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f8418007b70 tx=0x7f8418007e80 comp rx=0 tx=0).stop 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 shutdown_connections 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f840c0a11d0 0x7f840c0a3680 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8420100ce0 0x7f842019d340 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f84200ff160 0x7f8420069b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 --2- 172.21.15.93:0/2845483413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8420100350 0x7f842019acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 >> 172.21.15.93:0/2845483413 conn(0x7f84200fa780 msgr2=0x7f8420103830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 shutdown_connections 2023-12-14T18:24:42.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.794+0000 7f84279b3700 1 -- 172.21.15.93:0/2845483413 wait complete. 2023-12-14T18:24:42.807 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:42 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/890335528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.807 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:42 smithi093 ceph-mon[42070]: pgmap v227: 129 pgs: 3 creating+activating, 2 creating+peering, 108 active+clean, 16 unknown; 579 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2023-12-14T18:24:42.807 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:42 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/613343527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.807 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:42 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3337225571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-14T18:24:36.557875+0000', 'modified': '2023-12-14T18:24:40.995786+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24461}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6839/3753662808', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3753662808}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3753662808}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1} 2023-12-14T18:24:42.809 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-14T18:24:42.809 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-14T18:24:42.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:42 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/890335528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:42 smithi097 ceph-mon[43486]: pgmap v227: 129 pgs: 3 creating+activating, 2 creating+peering, 108 active+clean, 16 unknown; 579 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2023-12-14T18:24:42.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:42 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/613343527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:42 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3337225571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.930+0000 7f59bd19b700 1 -- 172.21.15.93:0/2601462755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b80ff130 msgr2=0x7f59b80ff540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.930+0000 7f59bd19b700 1 --2- 172.21.15.93:0/2601462755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b80ff130 0x7f59b80ff540 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f59a8009990 tx=0x7f59a8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.930+0000 7f59bd19b700 1 -- 172.21.15.93:0/2601462755 shutdown_connections 2023-12-14T18:24:42.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.930+0000 7f59bd19b700 1 --2- 172.21.15.93:0/2601462755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b80ff130 0x7f59b80ff540 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.930+0000 7f59bd19b700 1 --2- 172.21.15.93:0/2601462755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59b8100cb0 0x7f59b8107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.930+0000 7f59bd19b700 1 --2- 172.21.15.93:0/2601462755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b8100320 0x7f59b8100770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.930+0000 7f59bd19b700 1 -- 172.21.15.93:0/2601462755 >> 172.21.15.93:0/2601462755 conn(0x7f59b80fa750 msgr2=0x7f59b80fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.930+0000 7f59bd19b700 1 -- 172.21.15.93:0/2601462755 shutdown_connections 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.930+0000 7f59bd19b700 1 -- 172.21.15.93:0/2601462755 wait complete. 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59bd19b700 1 Processor -- start 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59bd19b700 1 -- start start 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59bd19b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b80ff130 0x7f59b8195f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59bd19b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b8100320 0x7f59b81964a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59bd19b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59b8100cb0 0x7f59b819d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59bd19b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b819dae0 con 0x7f59b8100320 2023-12-14T18:24:42.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59bd19b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b819dc20 con 0x7f59b80ff130 2023-12-14T18:24:42.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59bd19b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59b819ddb0 con 0x7f59b8100cb0 2023-12-14T18:24:42.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59b659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b8100320 0x7f59b81964a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59b659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b8100320 0x7f59b81964a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49252/0 (socket says 172.21.15.93:49252) 2023-12-14T18:24:42.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59b659c700 1 -- 172.21.15.93:0/3466254571 learned_addr learned my addr 172.21.15.93:0/3466254571 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:42.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b80ff130 0x7f59b8195f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.931+0000 7f59b759e700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59b8100cb0 0x7f59b819d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59b659c700 1 -- 172.21.15.93:0/3466254571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59b8100cb0 msgr2=0x7f59b819d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59b659c700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59b8100cb0 0x7f59b819d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59b659c700 1 -- 172.21.15.93:0/3466254571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b80ff130 msgr2=0x7f59b8195f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59b659c700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b80ff130 0x7f59b8195f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59b659c700 1 -- 172.21.15.93:0/3466254571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a80095f0 con 0x7f59b8100320 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59b659c700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b8100320 0x7f59b81964a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f59a000c9b0 tx=0x7f59a000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59affff700 1 -- 172.21.15.93:0/3466254571 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0007b00 con 0x7f59b8100320 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59affff700 1 -- 172.21.15.93:0/3466254571 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59a000ce50 con 0x7f59b8100320 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59bd19b700 1 -- 172.21.15.93:0/3466254571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59b810aac0 con 0x7f59b8100320 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59affff700 1 -- 172.21.15.93:0/3466254571 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a0018660 con 0x7f59b8100320 2023-12-14T18:24:42.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.932+0000 7f59bd19b700 1 -- 172.21.15.93:0/3466254571 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b810a930 con 0x7f59b8100320 2023-12-14T18:24:42.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.934+0000 7f59affff700 1 -- 172.21.15.93:0/3466254571 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f59a000f450 con 0x7f59b8100320 2023-12-14T18:24:42.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.935+0000 7f59affff700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f59a40a1150 0x7f59a40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.935+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f59a40a1150 0x7f59a40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.935+0000 7f59affff700 1 -- 172.21.15.93:0/3466254571 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f59a00cca40 con 0x7f59b8100320 2023-12-14T18:24:42.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.935+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f59a40a1150 0x7f59a40a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f59a800b5c0 tx=0x7f59a8005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59c9329b80 1 Processor -- start 2023-12-14T18:24:42.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59c9329b80 1 -- start start 2023-12-14T18:24:42.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a48c0 0x5575c67a4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 0x5575c67a5b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67a6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59c9329b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c6689590 con 0x5575c67a60c0 2023-12-14T18:24:42.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59c9329b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c66f8820 con 0x5575c67a5210 2023-12-14T18:24:42.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59c9329b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c66f1cb0 con 0x5575c67a48c0 2023-12-14T18:24:42.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67a6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67a6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49264/0 (socket says 172.21.15.93:49264) 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b759e700 1 -- 172.21.15.93:0/1788582447 learned_addr learned my addr 172.21.15.93:0/1788582447 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b659c700 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 0x5575c67a5b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a48c0 0x5575c67a4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b759e700 1 -- 172.21.15.93:0/1788582447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a48c0 msgr2=0x5575c67a4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b759e700 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a48c0 0x5575c67a4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b759e700 1 -- 172.21.15.93:0/1788582447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 msgr2=0x5575c67a5b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b759e700 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 0x5575c67a5b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b759e700 1 -- 172.21.15.93:0/1788582447 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575c6573360 con 0x5575c67a60c0 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.936+0000 7f59b659c700 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 0x5575c67a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59b759e700 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67a6530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f59b000a730 tx=0x7f59b00078d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f719fcbdb9dc1886 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59ad7fa700 1 -- 172.21.15.93:0/1788582447 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0018070 con 0x5575c67a60c0 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59ad7fa700 1 -- 172.21.15.93:0/1788582447 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b0010040 con 0x5575c67a60c0 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59ad7fa700 1 -- 172.21.15.93:0/1788582447 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b000f050 con 0x5575c67a60c0 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59c9329b80 1 -- 172.21.15.93:0/1788582447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 msgr2=0x5575c67a6530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59c9329b80 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67a6530 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f59b000a730 tx=0x7f59b00078d0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59c9329b80 1 -- 172.21.15.93:0/1788582447 shutdown_connections 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59c9329b80 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67a6530 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59c9329b80 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a48c0 0x5575c67a4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59c9329b80 1 --2- 172.21.15.93:0/1788582447 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 0x5575c67a5b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.937+0000 7f59c9329b80 1 -- 172.21.15.93:0/1788582447 >> 172.21.15.93:0/1788582447 conn(0x5575c6795c50 msgr2=0x5575c679b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 -- 172.21.15.93:0/1788582447 shutdown_connections 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 -- 172.21.15.93:0/1788582447 wait complete. 2023-12-14T18:24:42.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 -- start start 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 Processor -- start 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67ae4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a5210 0x5575c67ae9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a60c0 0x5575c67b5a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c668df40 con 0x5575c67a48c0 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c668dbe0 con 0x5575c67a60c0 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c6575650 con 0x5575c67a5210 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59c9329b80 10 client.0 _ll_register_callbacks cb 0x5575c66b18c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59b6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67ae4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59b6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67ae4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49266/0 (socket says 172.21.15.93:49266) 2023-12-14T18:24:42.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59b6d9d700 1 -- 172.21.15.93:0/3969792512 learned_addr learned my addr 172.21.15.93:0/3969792512 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59b659c700 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a5210 0x5575c67ae9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.938+0000 7f59b759e700 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a60c0 0x5575c67b5a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59b6d9d700 1 -- 172.21.15.93:0/3969792512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a5210 msgr2=0x5575c67ae9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a5210 0x5575c67ae9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59b6d9d700 1 -- 172.21.15.93:0/3969792512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a60c0 msgr2=0x5575c67b5a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a60c0 0x5575c67b5a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59b6d9d700 1 -- 172.21.15.93:0/3969792512 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59b000a410 con 0x5575c67a48c0 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67ae4a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f59a8015000 tx=0x7f59a801a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcbc55762e2fcaaa server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f599e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f599e7fc700 1 -- 172.21.15.93:0/3969792512 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a800bea0 con 0x5575c67a48c0 2023-12-14T18:24:42.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59c9329b80 1 -- 172.21.15.93:0/3969792512 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5575c6793260 con 0x5575c67a48c0 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f599e7fc700 1 -- 172.21.15.93:0/3969792512 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f59a800f460 con 0x5575c67a48c0 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f599e7fc700 1 -- 172.21.15.93:0/3969792512 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a800f620 con 0x5575c67a48c0 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59c9329b80 10 client.14739 fetch_fsmap learned FSMap version 17 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59c9329b80 1 -- 172.21.15.93:0/3969792512 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5575c66b8120 con 0x5575c67a48c0 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f599e7fc700 1 -- 172.21.15.93:0/3969792512 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5575c6793260 con 0x5575c67a48c0 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f599e7fc700 1 -- 172.21.15.93:0/3969792512 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f59a8030720 con 0x5575c67a48c0 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59c9329b80 10 client.14739 fetch_fsmap finished waiting for FSMap version 17 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59c9329b80 10 client.14739 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59c9329b80 20 client.14739 populate_metadata read hostname 'smithi093' 2023-12-14T18:24:42.953 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5575c67b9720 0x5575c67b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59c9329b80 4 client.14739 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.939+0000 7f59c9329b80 1 -- 172.21.15.93:0/3969792512 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5575c66eac90 con 0x5575c67b9720 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.940+0000 7f59b659c700 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5575c67b9720 0x5575c67b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.940+0000 7f59b659c700 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5575c67b9720 0x5575c67b8b20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7503578808c3a595 server_cookie=5bd2e7e277a153c2 in_seq=0 out_seq=0 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.940+0000 7f599e7fc700 10 client.14739 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f599e7fc700 1 -- 172.21.15.93:0/3969792512 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5575c66eac90 con 0x5575c67b9720 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f599e7fc700 10 client.14739 handle_command_reply: tid=0 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 client.14739 shutdown 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 -- 172.21.15.93:0/3969792512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 msgr2=0x5575c67ae4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67ae4a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f59a8015000 tx=0x7f59a801a050 comp rx=0 tx=0).stop 2023-12-14T18:24:42.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 -- 172.21.15.93:0/3969792512 shutdown_connections 2023-12-14T18:24:42.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5575c67b9720 0x5575c67b8b20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67ae4a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f599e7fc700 0 client.14739 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:24:42.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a60c0 0x5575c67b5a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3969792512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a5210 0x5575c67ae9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 -- 172.21.15.93:0/3969792512 >> 172.21.15.93:0/3969792512 conn(0x5575c6795c50 msgr2=0x5575c6799ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 -- 172.21.15.93:0/3969792512 shutdown_connections 2023-12-14T18:24:42.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 1 -- 172.21.15.93:0/3969792512 wait complete. 2023-12-14T18:24:42.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.941+0000 7f59c9329b80 20 client.14739 trim_cache size 0 max 16384 2023-12-14T18:24:42.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.958+0000 7f59c9329b80 1 Processor -- start 2023-12-14T18:24:42.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59c9329b80 1 -- start start 2023-12-14T18:24:42.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a48c0 0x5575c67ad240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a5210 0x5575c67b59e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67ac450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59c9329b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c669b580 con 0x5575c67a60c0 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59c9329b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c66b0290 con 0x5575c67a48c0 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59c9329b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c67b9fc0 con 0x5575c67a5210 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67ac450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b759e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67ac450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49280/0 (socket says 172.21.15.93:49280) 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b759e700 1 -- 172.21.15.93:0/3743301245 learned_addr learned my addr 172.21.15.93:0/3743301245 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/3743301245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a48c0 0x5575c67ad240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b759e700 1 -- 172.21.15.93:0/3743301245 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a5210 msgr2=0x5575c67b59e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:24:42.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b759e700 1 --2- 172.21.15.93:0/3743301245 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a5210 0x5575c67b59e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b759e700 1 -- 172.21.15.93:0/3743301245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a48c0 msgr2=0x5575c67ad240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b759e700 1 --2- 172.21.15.93:0/3743301245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a48c0 0x5575c67ad240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b759e700 1 -- 172.21.15.93:0/3743301245 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5575c67af3d0 con 0x5575c67a60c0 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59b759e700 1 --2- 172.21.15.93:0/3743301245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67ac450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5575c679cd30 tx=0x7f59b000a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b23bde942836d6cb server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59adffb700 1 -- 172.21.15.93:0/3743301245 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0018030 con 0x5575c67a60c0 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.959+0000 7f59adffb700 1 -- 172.21.15.93:0/3743301245 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b0019040 con 0x5575c67a60c0 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59adffb700 1 -- 172.21.15.93:0/3743301245 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0004140 con 0x5575c67a60c0 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 -- 172.21.15.93:0/3743301245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 msgr2=0x5575c67ac450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3743301245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67ac450 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x5575c679cd30 tx=0x7f59b000a380 comp rx=0 tx=0).stop 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 -- 172.21.15.93:0/3743301245 shutdown_connections 2023-12-14T18:24:42.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3743301245 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a60c0 0x5575c67ac450 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3743301245 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a48c0 0x5575c67ad240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 --2- 172.21.15.93:0/3743301245 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a5210 0x5575c67b59e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 -- 172.21.15.93:0/3743301245 >> 172.21.15.93:0/3743301245 conn(0x5575c6795c50 msgr2=0x5575c679b230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 -- 172.21.15.93:0/3743301245 shutdown_connections 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 -- 172.21.15.93:0/3743301245 wait complete. 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.960+0000 7f59c9329b80 1 -- start start 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59c9329b80 1 Processor -- start 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67b07e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 0x5575c67b0d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59c9329b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a60c0 0x5575c680d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59c9329b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c67b1400 con 0x5575c67a48c0 2023-12-14T18:24:42.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59c9329b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c680db20 con 0x5575c67a5210 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59c9329b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5575c680dc60 con 0x5575c67a60c0 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59c9329b80 10 client.0 _ll_register_callbacks cb 0x7f59a800ba80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67b07e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b6d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67b07e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49292/0 (socket says 172.21.15.93:49292) 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b6d9d700 1 -- 172.21.15.93:0/74041052 learned_addr learned my addr 172.21.15.93:0/74041052 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b659c700 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 0x5575c67b0d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b759e700 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a60c0 0x5575c680d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b6d9d700 1 -- 172.21.15.93:0/74041052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a60c0 msgr2=0x5575c680d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a60c0 0x5575c680d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b6d9d700 1 -- 172.21.15.93:0/74041052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 msgr2=0x5575c67b0d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:24:42.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 0x5575c67b0d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b6d9d700 1 -- 172.21.15.93:0/74041052 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5575c6573360 con 0x5575c67a48c0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59b6d9d700 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67b07e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59a8030fd0 tx=0x7f59a8009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4727f84a859d349f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f599e7fc700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f599e7fc700 1 -- 172.21.15.93:0/74041052 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a8030d70 con 0x5575c67a48c0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f599e7fc700 1 -- 172.21.15.93:0/74041052 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f59a8030630 con 0x5575c67a48c0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.961+0000 7f59c9329b80 1 -- 172.21.15.93:0/74041052 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f59b0010040 con 0x5575c67a48c0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f599e7fc700 1 -- 172.21.15.93:0/74041052 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59a80307f0 con 0x5575c67a48c0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f59c9329b80 10 client.14745 fetch_fsmap learned FSMap version 17 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f59c9329b80 1 -- 172.21.15.93:0/74041052 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) v3 -- 0x5575c651fb10 con 0x5575c67a48c0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f599e7fc700 1 -- 172.21.15.93:0/74041052 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59b0010040 con 0x5575c67a48c0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f599e7fc700 1 -- 172.21.15.93:0/74041052 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 17) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f59a8003890 con 0x5575c67a48c0 2023-12-14T18:24:42.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f59c9329b80 10 client.14745 fetch_fsmap finished waiting for FSMap version 17 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f59c9329b80 10 client.14745 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f59c9329b80 20 client.14745 populate_metadata read hostname 'smithi093' 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f59c9329b80 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5575c6812520 0x5575c680eed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f59c9329b80 4 client.14745 mds_command: new command op to 24461 tid=0[{"prefix": "status"}] 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f59c9329b80 1 -- 172.21.15.93:0/74041052 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5575c66ea9b0 con 0x5575c6812520 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.962+0000 7f59b659c700 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5575c6812520 0x5575c680eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f599e7fc700 10 client.14745 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59b659c700 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5575c6812520 0x5575c680eed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b122cd680340d1b server_cookie=c34d881ca113468f in_seq=0 out_seq=0 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f599e7fc700 1 -- 172.21.15.93:0/74041052 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x5575c66ea9b0 con 0x5575c6812520 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f599e7fc700 10 client.14745 handle_command_reply: tid=0 2023-12-14T18:24:42.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 client.14745 shutdown 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 -- 172.21.15.93:0/74041052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 msgr2=0x5575c67b07e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67b07e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f59a8030fd0 tx=0x7f59a8009340 comp rx=0 tx=0).stop 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 -- 172.21.15.93:0/74041052 shutdown_connections 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5575c6812520 0x5575c680eed0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5575c67a48c0 0x5575c67b07e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f599e7fc700 0 client.14745 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5575c67a60c0 0x5575c680d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 --2- 172.21.15.93:0/74041052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5575c67a5210 0x5575c67b0d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 -- 172.21.15.93:0/74041052 >> 172.21.15.93:0/74041052 conn(0x5575c6795c50 msgr2=0x5575c67a8f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 -- 172.21.15.93:0/74041052 shutdown_connections 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 1 -- 172.21.15.93:0/74041052 wait complete. 2023-12-14T18:24:42.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.963+0000 7f59c9329b80 20 client.14745 trim_cache size 0 max 16384 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 0, 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24461, 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 6.4034378060000003, 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 17, 2023-12-14T18:24:42.975 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 84, 2023-12-14T18:24:42.976 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 83, 2023-12-14T18:24:42.976 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 11.875849718 2023-12-14T18:24:42.976 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:24:42.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 -- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f59a40a1150 msgr2=0x7f59a40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f59a40a1150 0x7f59a40a3600 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f59a800b5c0 tx=0x7f59a8005960 comp rx=0 tx=0).stop 2023-12-14T18:24:42.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 -- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b8100320 msgr2=0x7f59b81964a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:42.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b8100320 0x7f59b81964a0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f59a000c9b0 tx=0x7f59a000ccc0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 -- 172.21.15.93:0/3466254571 shutdown_connections 2023-12-14T18:24:42.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f59a40a1150 0x7f59a40a3600 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f59b8100cb0 0x7f59b819d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f59b80ff130 0x7f59b8195f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.977 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 --2- 172.21.15.93:0/3466254571 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59b8100320 0x7f59b81964a0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:42.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 -- 172.21.15.93:0/3466254571 >> 172.21.15.93:0/3466254571 conn(0x7f59b80fa750 msgr2=0x7f59b80fc900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:42.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 -- 172.21.15.93:0/3466254571 shutdown_connections 2023-12-14T18:24:42.978 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:42.964+0000 7f59bd19b700 1 -- 172.21.15.93:0/3466254571 wait complete. 2023-12-14T18:24:42.979 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-14T18:24:42.979 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:24:42.979 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph fs ls 2023-12-14T18:24:42.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:42 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/890335528' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:42 smithi160 ceph-mon[43840]: pgmap v227: 129 pgs: 3 creating+activating, 2 creating+peering, 108 active+clean, 16 unknown; 579 KiB data, 75 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s wr, 9 op/s 2023-12-14T18:24:42.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:42 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/613343527' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:42.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:42 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3337225571' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:24:43.285 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:24:43.759 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:43 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2845483413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:43.995 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:43 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2845483413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:44.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:43 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2845483413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 -- 172.21.15.93:0/67284971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 msgr2=0x7fd1100ff900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 --2- 172.21.15.93:0/67284971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 0x7fd1100ff900 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fd104009990 tx=0x7fd104009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 -- 172.21.15.93:0/67284971 shutdown_connections 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 --2- 172.21.15.93:0/67284971 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 0x7fd1100ff900 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 --2- 172.21.15.93:0/67284971 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11010adf0 0x7fd11010d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 --2- 172.21.15.93:0/67284971 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1100ffe40 0x7fd11010a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 -- 172.21.15.93:0/67284971 >> 172.21.15.93:0/67284971 conn(0x7fd110074b30 msgr2=0x7fd110074f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 -- 172.21.15.93:0/67284971 shutdown_connections 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 -- 172.21.15.93:0/67284971 wait complete. 2023-12-14T18:24:44.337 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.333+0000 7fd115940700 1 Processor -- start 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd115940700 1 -- start start 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd115940700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 0x7fd110198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd115940700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1100ffe40 0x7fd110198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd115940700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11010adf0 0x7fd11019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd115940700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11019d5f0 con 0x7fd1100ff530 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd115940700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11019d760 con 0x7fd1100ffe40 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd115940700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd11019d920 con 0x7fd11010adf0 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 0x7fd110198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 0x7fd110198630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49304/0 (socket says 172.21.15.93:49304) 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10effd700 1 -- 172.21.15.93:0/3511748337 learned_addr learned my addr 172.21.15.93:0/3511748337 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10e7fc700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1100ffe40 0x7fd110198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:44.338 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10f7fe700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11010adf0 0x7fd11019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10effd700 1 -- 172.21.15.93:0/3511748337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11010adf0 msgr2=0x7fd11019cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10effd700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11010adf0 0x7fd11019cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10effd700 1 -- 172.21.15.93:0/3511748337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1100ffe40 msgr2=0x7fd110198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10effd700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1100ffe40 0x7fd110198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10effd700 1 -- 172.21.15.93:0/3511748337 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1040095f0 con 0x7fd1100ff530 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10f7fe700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11010adf0 0x7fd11019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd10effd700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 0x7fd110198630 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd1040095c0 tx=0x7fd10400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd11493e700 1 -- 172.21.15.93:0/3511748337 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd10401d070 con 0x7fd1100ff530 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.334+0000 7fd11493e700 1 -- 172.21.15.93:0/3511748337 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd10400fb50 con 0x7fd1100ff530 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.335+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd110110420 con 0x7fd1100ff530 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.335+0000 7fd11493e700 1 -- 172.21.15.93:0/3511748337 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1040173f0 con 0x7fd1100ff530 2023-12-14T18:24:44.339 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.335+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd110110290 con 0x7fd1100ff530 2023-12-14T18:24:44.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.336+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1101105b0 con 0x7fd1100ff530 2023-12-14T18:24:44.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.337+0000 7fd11493e700 1 -- 172.21.15.93:0/3511748337 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd10400fcc0 con 0x7fd1100ff530 2023-12-14T18:24:44.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.337+0000 7fd11493e700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0f80a1150 0x7fd0f80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:44.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.337+0000 7fd10e7fc700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0f80a1150 0x7fd0f80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:44.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.337+0000 7fd11493e700 1 -- 172.21.15.93:0/3511748337 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd1040cdba0 con 0x7fd1100ff530 2023-12-14T18:24:44.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.338+0000 7fd10e7fc700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0f80a1150 0x7fd0f80a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd110199c50 tx=0x7fd10000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:44.343 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.341+0000 7fd11493e700 1 -- 172.21.15.93:0/3511748337 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd10405c730 con 0x7fd1100ff530 2023-12-14T18:24:44.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.578+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd110199a90 con 0x7fd1100ff530 2023-12-14T18:24:44.581 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.579+0000 7fd11493e700 1 -- 172.21.15.93:0/3511748337 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+88 (secure 0 0 0) 0x7fd104097190 con 0x7fd1100ff530 2023-12-14T18:24:44.581 INFO:teuthology.orchestra.run.smithi093.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-14T18:24:44.583 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.581+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0f80a1150 msgr2=0x7fd0f80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:44.583 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.581+0000 7fd115940700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0f80a1150 0x7fd0f80a3600 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd110199c50 tx=0x7fd10000b410 comp rx=0 tx=0).stop 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.581+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 msgr2=0x7fd110198630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.581+0000 7fd115940700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 0x7fd110198630 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fd1040095c0 tx=0x7fd10400f710 comp rx=0 tx=0).stop 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.582+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 shutdown_connections 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.582+0000 7fd115940700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0f80a1150 0x7fd0f80a3600 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.582+0000 7fd115940700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd1100ff530 0x7fd110198630 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.582+0000 7fd115940700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd11010adf0 0x7fd11019cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.582+0000 7fd115940700 1 --2- 172.21.15.93:0/3511748337 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd1100ffe40 0x7fd110198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.582+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 >> 172.21.15.93:0/3511748337 conn(0x7fd110074b30 msgr2=0x7fd1100fe520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.582+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 shutdown_connections 2023-12-14T18:24:44.584 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:44.582+0000 7fd115940700 1 -- 172.21.15.93:0/3511748337 wait complete. 2023-12-14T18:24:44.921 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:44 smithi093 ceph-mon[42070]: pgmap v228: 129 pgs: 3 creating+activating, 2 creating+peering, 121 active+clean, 3 unknown; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2023-12-14T18:24:44.921 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:44 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3511748337' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:24:44.963 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-14T18:24:44.963 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-14T18:24:44.964 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2023-12-14T18:24:44.964 INFO:tasks.cephfs.mount:self.client.name = client.admin 2023-12-14T18:24:44.964 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:24:44.964 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-14T18:24:44.964 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-14T18:24:44.964 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-14T18:24:44.964 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2023-12-14T18:24:44.964 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:44.964 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2023-12-14T18:24:44.983 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-14T18:24:44.983 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-14T18:24:44.983 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2023-12-14T18:24:44.983 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:24:44.983 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f0 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f1 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-14T18:24:44.984 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 42067sec preferred_lft 42067sec 2023-12-14T18:24:44.985 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::1355:f7c1:e067:88d9/64 scope link noprefixroute 2023-12-14T18:24:44.985 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:24:44.985 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-14T18:24:44.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:44.985 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:24:44.986 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:24:44.986 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add name ceph-brx type bridge 2023-12-14T18:24:44.986 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr flush dev ceph-brx 2023-12-14T18:24:44.986 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set ceph-brx up 2023-12-14T18:24:44.986 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-14T18:24:44.986 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:24:44.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:44 smithi160 ceph-mon[43840]: pgmap v228: 129 pgs: 3 creating+activating, 2 creating+peering, 121 active+clean, 3 unknown; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2023-12-14T18:24:44.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:44 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3511748337' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:24:45.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:44 smithi097 ceph-mon[43486]: pgmap v228: 129 pgs: 3 creating+activating, 2 creating+peering, 121 active+clean, 3 unknown; 579 KiB data, 76 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s wr, 6 op/s 2023-12-14T18:24:45.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:44 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3511748337' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:24:45.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:45.155 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-14T18:24:45.183 INFO:teuthology.orchestra.run.smithi093.stdout:1 2023-12-14T18:24:45.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:45.184 DEBUG:teuthology.orchestra.run.smithi093:> route 2023-12-14T18:24:45.324 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2023-12-14T18:24:45.324 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-14T18:24:45.325 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-14T18:24:45.325 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-14T18:24:45.325 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-14T18:24:45.326 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:45.326 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:24:45.326 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:24:45.326 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-14T18:24:45.326 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-14T18:24:45.326 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-14T18:24:45.327 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:24:45.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:45.574 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-12-14T18:24:45.591 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:45.592 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list-id 2023-12-14T18:24:45.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:45.649 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:24:45.649 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:24:45.649 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-14T18:24:45.649 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2023-12-14T18:24:45.649 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:24:45.747 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2023-12-14T18:24:45.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:45.747 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:24:45.747 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:24:45.747 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2023-12-14T18:24:45.747 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-14T18:24:45.747 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2023-12-14T18:24:45.748 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2023-12-14T18:24:45.748 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2023-12-14T18:24:45.748 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:24:45.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:45.950 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:24:45.950 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:24:45.950 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.0 up 2023-12-14T18:24:45.950 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set dev brx.0 master ceph-brx 2023-12-14T18:24:45.950 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:24:46.048 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2023-12-14T18:24:46.048 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-14T18:24:46.048 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:24:46.065 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2023-12-14T18:24:46.065 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-14T18:24:46.065 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:24:46.121 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-12-14T18:24:46.189 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-14T18:24:46.189 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-14T18:24:46.256 INFO:teuthology.orchestra.run.smithi093.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-14T18:24:46.257 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-14T18:24:46.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:46.258 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-12-14T18:24:46.315 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-12-14T18:24:46.315 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2023-12-14T18:24:46.358 DEBUG:teuthology.orchestra.run.smithi093:> sudo modprobe fuse 2023-12-14T18:24:46.384 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-12-14T18:24:46.384 DEBUG:teuthology.orchestra.run.smithi093:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-12-14T18:24:46.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi093.stderr:2023-12-14T18:24:46.445+0000 7f4dab6c65c0 -1 init, newargv = 0x55bef4f27190 newargc=15 2023-12-14T18:24:46.447 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi093.stderr:ceph-fuse[78737]: starting ceph client 2023-12-14T18:24:46.452 INFO:teuthology.orchestra.run.smithi093.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-12-14T18:24:46.453 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-12-14T18:24:46.454 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:46.454 DEBUG:teuthology.orchestra.run.smithi093:> ls /sys/fs/fuse/connections 2023-12-14T18:24:46.454 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi093.stderr:ceph-fuse[78737]: starting fuse 2023-12-14T18:24:46.510 INFO:teuthology.orchestra.run.smithi093.stdout:124 2023-12-14T18:24:46.510 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [124] 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> import glob 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> import re 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> import os 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> import subprocess 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> def _find_admin_socket(client_name): 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> files = glob.glob(asok_path) 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> # Given a non-glob path, it better be there 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> if "*" not in asok_path: 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> assert(len(files) == 1) 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> return files[0] 2023-12-14T18:24:46.511 DEBUG:teuthology.orchestra.run.smithi093:> 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> for f in files: 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> if os.path.exists("/proc/{0}".format(pid)): 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> contents = proc_f.read() 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> if mountpoint in contents: 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> return f 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> print(_find_admin_socket("client.admin")) 2023-12-14T18:24:46.512 DEBUG:teuthology.orchestra.run.smithi093:> ' 2023-12-14T18:24:46.634 INFO:teuthology.orchestra.run.smithi093.stdout:/var/run/ceph/ceph-client.admin.78737.asok 2023-12-14T18:24:46.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:46 smithi093 ceph-mon[42070]: pgmap v229: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 643 B/s wr, 5 op/s 2023-12-14T18:24:46.816 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.78737.asok 2023-12-14T18:24:46.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:46.817 DEBUG:teuthology.orchestra.run.smithi093:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.78737.asok status 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "metadata": { 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_sha1": "1faa353368ad4f5aeb56d7107b638336f58659ed", 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_version": "ceph version 18.0.0-3905-g1faa3533 (1faa353368ad4f5aeb56d7107b638336f58659ed) reef (dev)", 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "entity_id": "admin", 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": "smithi093", 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "pid": "78737", 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "root": "/" 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "dentry_count": 0, 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "dentry_pinned_count": 0, 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 14757, 2023-12-14T18:24:46.975 INFO:teuthology.orchestra.run.smithi093.stdout: "inst": { 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "name": { 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "client", 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "num": 14757 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": { 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "192.168.0.1:0", 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 213279042 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": { 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "192.168.0.1:0", 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 213279042 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:24:46.976 INFO:teuthology.orchestra.run.smithi093.stdout: "inst_str": "client.14757 192.168.0.1:0/213279042", 2023-12-14T18:24:46.977 INFO:teuthology.orchestra.run.smithi093.stdout: "addr_str": "192.168.0.1:0/213279042", 2023-12-14T18:24:46.977 INFO:teuthology.orchestra.run.smithi093.stdout: "inode_count": 1, 2023-12-14T18:24:46.977 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_epoch": 17, 2023-12-14T18:24:46.977 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_epoch": 84, 2023-12-14T18:24:46.977 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_epoch_barrier": 0, 2023-12-14T18:24:46.977 INFO:teuthology.orchestra.run.smithi093.stdout: "blocklisted": false, 2023-12-14T18:24:46.977 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs" 2023-12-14T18:24:46.977 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:24:46.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:46.985 DEBUG:teuthology.orchestra.run.smithi093:> stat --file-system '--printf=%T 2023-12-14T18:24:46.985 DEBUG:teuthology.orchestra.run.smithi093:> ' -- /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:24:47.002 INFO:teuthology.orchestra.run.smithi093.stdout:fuseblk 2023-12-14T18:24:47.002 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:24:47.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:47.002 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:24:47.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:47.071 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.admin && exec bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2023-12-14T18:24:47.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:46 smithi097 ceph-mon[43486]: pgmap v229: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 643 B/s wr, 5 op/s 2023-12-14T18:24:47.207 INFO:teuthology.orchestra.run.smithi093.stdout:# file: . 2023-12-14T18:24:47.207 INFO:teuthology.orchestra.run.smithi093.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-12-14T18:24:47.207 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:47.208 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:24:47.208 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:24:47.226 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:24:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:46 smithi160 ceph-mon[43840]: pgmap v229: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 643 B/s wr, 5 op/s 2023-12-14T18:24:47.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.386+0000 7f748cd06700 1 -- 172.21.15.93:0/4040779686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 msgr2=0x7f7488108340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.386+0000 7f748cd06700 1 --2- 172.21.15.93:0/4040779686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 0x7f7488108340 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f748000b0a0 tx=0x7f748000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.386+0000 7f748cd06700 1 -- 172.21.15.93:0/4040779686 shutdown_connections 2023-12-14T18:24:47.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.386+0000 7f748cd06700 1 --2- 172.21.15.93:0/4040779686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 0x7f7488108340 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.386+0000 7f748cd06700 1 --2- 172.21.15.93:0/4040779686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7488100ca0 0x7f7488103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.386+0000 7f748cd06700 1 --2- 172.21.15.93:0/4040779686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74881035c0 0x7f74881059a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.386+0000 7f748cd06700 1 -- 172.21.15.93:0/4040779686 >> 172.21.15.93:0/4040779686 conn(0x7f74880fa700 msgr2=0x7f74880fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:47.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.386+0000 7f748cd06700 1 -- 172.21.15.93:0/4040779686 shutdown_connections 2023-12-14T18:24:47.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.386+0000 7f748cd06700 1 -- 172.21.15.93:0/4040779686 wait complete. 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748cd06700 1 Processor -- start 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748cd06700 1 -- start start 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748cd06700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7488100ca0 0x7f7488100610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748cd06700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74881035c0 0x7f74880fec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748cd06700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 0x7f74880ff190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748cd06700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880ff930 con 0x7f7488105ee0 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748cd06700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880ffaa0 con 0x7f74881035c0 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748cd06700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74881a0030 con 0x7f7488100ca0 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f7486d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 0x7f74880ff190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f7486d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 0x7f74880ff190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49324/0 (socket says 172.21.15.93:49324) 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f7486d9d700 1 -- 172.21.15.93:0/1152670241 learned_addr learned my addr 172.21.15.93:0/1152670241 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:47.391 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748659c700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7488100ca0 0x7f7488100610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f7485d9b700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74881035c0 0x7f74880fec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f7486d9d700 1 -- 172.21.15.93:0/1152670241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7488100ca0 msgr2=0x7f7488100610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f7486d9d700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7488100ca0 0x7f7488100610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f7486d9d700 1 -- 172.21.15.93:0/1152670241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74881035c0 msgr2=0x7f74880fec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f7486d9d700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74881035c0 0x7f74880fec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f7486d9d700 1 -- 172.21.15.93:0/1152670241 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7480009d00 con 0x7f7488105ee0 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.387+0000 7f748659c700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7488100ca0 0x7f7488100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.388+0000 7f7486d9d700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 0x7f74880ff190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f748000b070 tx=0x7f7480004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.388+0000 7f74777fe700 1 -- 172.21.15.93:0/1152670241 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748000e040 con 0x7f7488105ee0 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.388+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f748810bbe0 con 0x7f7488105ee0 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.388+0000 7f74777fe700 1 -- 172.21.15.93:0/1152670241 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f748001bd20 con 0x7f7488105ee0 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.388+0000 7f74777fe700 1 -- 172.21.15.93:0/1152670241 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7480005910 con 0x7f7488105ee0 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.388+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f748810ba50 con 0x7f7488105ee0 2023-12-14T18:24:47.392 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.389+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74880fc270 con 0x7f7488105ee0 2023-12-14T18:24:47.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.390+0000 7f74777fe700 1 -- 172.21.15.93:0/1152670241 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f7480005040 con 0x7f7488105ee0 2023-12-14T18:24:47.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.391+0000 7f74777fe700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f74700a1150 0x7f74700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:47.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.391+0000 7f748659c700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f74700a1150 0x7f74700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:47.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.391+0000 7f74777fe700 1 -- 172.21.15.93:0/1152670241 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f74800cf0a0 con 0x7f7488105ee0 2023-12-14T18:24:47.393 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.391+0000 7f748659c700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f74700a1150 0x7f74700a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f747c005950 tx=0x7f747c005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:47.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.394+0000 7f74777fe700 1 -- 172.21.15.93:0/1152670241 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7480098690 con 0x7f7488105ee0 2023-12-14T18:24:47.585 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.584+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7488067160 con 0x7f7488105ee0 2023-12-14T18:24:47.587 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.585+0000 7f74777fe700 1 -- 172.21.15.93:0/1152670241 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7f7480009500 con 0x7f7488105ee0 2023-12-14T18:24:47.587 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:47.587 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.587+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f74700a1150 msgr2=0x7f74700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.587+0000 7f748cd06700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f74700a1150 0x7f74700a3600 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f747c005950 tx=0x7f747c005f50 comp rx=0 tx=0).stop 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.587+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 msgr2=0x7f74880ff190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.587+0000 7f748cd06700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 0x7f74880ff190 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f748000b070 tx=0x7f7480004db0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.587+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 shutdown_connections 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.587+0000 7f748cd06700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f74700a1150 0x7f74700a3600 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.587+0000 7f748cd06700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7488105ee0 0x7f74880ff190 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.587+0000 7f748cd06700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7488100ca0 0x7f7488100610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.587+0000 7f748cd06700 1 --2- 172.21.15.93:0/1152670241 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f74881035c0 0x7f74880fec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.588+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 >> 172.21.15.93:0/1152670241 conn(0x7f74880fa700 msgr2=0x7f7488101900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.588+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 shutdown_connections 2023-12-14T18:24:47.590 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.588+0000 7f748cd06700 1 -- 172.21.15.93:0/1152670241 wait complete. 2023-12-14T18:24:47.602 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:24:34.407045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:24:35.728548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:24:38.796189+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:24:47.602 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 -- 172.21.15.93:0/379280696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 msgr2=0x7f27881075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 --2- 172.21.15.93:0/379280696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 0x7f27881075a0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f278000b0a0 tx=0x7f278000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 -- 172.21.15.93:0/379280696 shutdown_connections 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 --2- 172.21.15.93:0/379280696 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 0x7f27881075a0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 --2- 172.21.15.93:0/379280696 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f27880ff160 0x7f27880ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 --2- 172.21.15.93:0/379280696 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2788100350 0x7f27881007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 -- 172.21.15.93:0/379280696 >> 172.21.15.93:0/379280696 conn(0x7f27880fa780 msgr2=0x7f27880fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 -- 172.21.15.93:0/379280696 shutdown_connections 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 -- 172.21.15.93:0/379280696 wait complete. 2023-12-14T18:24:47.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.725+0000 7f278f9d4700 1 Processor -- start 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278f9d4700 1 -- start start 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278f9d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27880ff160 0x7f2788196040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278f9d4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2788100350 0x7f2788196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278f9d4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 0x7f278819d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278f9d4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278819dbc0 con 0x7f2788100ce0 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278f9d4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278819dd00 con 0x7f2788100350 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278f9d4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f278819de90 con 0x7f27880ff160 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278df71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 0x7f278819d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278df71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 0x7f278819d5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49340/0 (socket says 172.21.15.93:49340) 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278df71700 1 -- 172.21.15.93:0/3320332509 learned_addr learned my addr 172.21.15.93:0/3320332509 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278cf6f700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2788100350 0x7f2788196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278d770700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27880ff160 0x7f2788196040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278df71700 1 -- 172.21.15.93:0/3320332509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27880ff160 msgr2=0x7f2788196040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278df71700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27880ff160 0x7f2788196040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.730 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278df71700 1 -- 172.21.15.93:0/3320332509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2788100350 msgr2=0x7f2788196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278df71700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2788100350 0x7f2788196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278df71700 1 -- 172.21.15.93:0/3320332509 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2780009d00 con 0x7f2788100ce0 2023-12-14T18:24:47.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278df71700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 0x7f278819d5f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f278000b780 tx=0x7f278000bd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:47.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f277e7fc700 1 -- 172.21.15.93:0/3320332509 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278000e040 con 0x7f2788100ce0 2023-12-14T18:24:47.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f277e7fc700 1 -- 172.21.15.93:0/3320332509 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2780004e60 con 0x7f2788100ce0 2023-12-14T18:24:47.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f278810aad0 con 0x7f2788100ce0 2023-12-14T18:24:47.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f277e7fc700 1 -- 172.21.15.93:0/3320332509 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278001b6b0 con 0x7f2788100ce0 2023-12-14T18:24:47.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.726+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f278810a940 con 0x7f2788100ce0 2023-12-14T18:24:47.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.728+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27881071a0 con 0x7f2788100ce0 2023-12-14T18:24:47.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.729+0000 7f277e7fc700 1 -- 172.21.15.93:0/3320332509 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f278001b810 con 0x7f2788100ce0 2023-12-14T18:24:47.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.729+0000 7f277e7fc700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f27740a1150 0x7f27740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:47.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.729+0000 7f277e7fc700 1 -- 172.21.15.93:0/3320332509 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f27800ce4d0 con 0x7f2788100ce0 2023-12-14T18:24:47.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.729+0000 7f278d770700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f27740a1150 0x7f27740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:47.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.730+0000 7f278d770700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f27740a1150 0x7f27740a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f27881001b0 tx=0x7f2778009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:47.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.733+0000 7f277e7fc700 1 -- 172.21.15.93:0/3320332509 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2780097ac0 con 0x7f2788100ce0 2023-12-14T18:24:47.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.922+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f278802cfe0 con 0x7f2788100ce0 2023-12-14T18:24:47.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.922+0000 7f277e7fc700 1 -- 172.21.15.93:0/3320332509 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7f278009b0e0 con 0x7f2788100ce0 2023-12-14T18:24:47.924 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:24:47.925 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:47 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1152670241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f27740a1150 msgr2=0x7f27740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f27740a1150 0x7f27740a3600 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f27881001b0 tx=0x7f2778009260 comp rx=0 tx=0).stop 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 msgr2=0x7f278819d5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 0x7f278819d5f0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f278000b780 tx=0x7f278000bd00 comp rx=0 tx=0).stop 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 shutdown_connections 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f27740a1150 0x7f27740a3600 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2788100ce0 0x7f278819d5f0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f27880ff160 0x7f2788196040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 --2- 172.21.15.93:0/3320332509 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2788100350 0x7f2788196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:47.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 >> 172.21.15.93:0/3320332509 conn(0x7f27880fa780 msgr2=0x7f27880fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:47.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 shutdown_connections 2023-12-14T18:24:47.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:47.925+0000 7f278f9d4700 1 -- 172.21.15.93:0/3320332509 wait complete. 2023-12-14T18:24:47.938 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:24:48.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.060+0000 7f25c8a57700 1 -- 172.21.15.93:0/1599325311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 msgr2=0x7f25c41059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.062 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.060+0000 7f25c8a57700 1 --2- 172.21.15.93:0/1599325311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 0x7f25c41059a0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009990 tx=0x7f25b8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.061+0000 7f25c8a57700 1 -- 172.21.15.93:0/1599325311 shutdown_connections 2023-12-14T18:24:48.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.061+0000 7f25c8a57700 1 --2- 172.21.15.93:0/1599325311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25c4105ee0 0x7f25c4108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.061+0000 7f25c8a57700 1 --2- 172.21.15.93:0/1599325311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25c4100ca0 0x7f25c4103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.061+0000 7f25c8a57700 1 --2- 172.21.15.93:0/1599325311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 0x7f25c41059a0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.061+0000 7f25c8a57700 1 -- 172.21.15.93:0/1599325311 >> 172.21.15.93:0/1599325311 conn(0x7f25c40fa740 msgr2=0x7f25c40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:48.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.061+0000 7f25c8a57700 1 -- 172.21.15.93:0/1599325311 shutdown_connections 2023-12-14T18:24:48.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.061+0000 7f25c8a57700 1 -- 172.21.15.93:0/1599325311 wait complete. 2023-12-14T18:24:48.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.061+0000 7f25c8a57700 1 Processor -- start 2023-12-14T18:24:48.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.061+0000 7f25c8a57700 1 -- start start 2023-12-14T18:24:48.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c8a57700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25c4100ca0 0x7f25c4195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c8a57700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 0x7f25c41964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c8a57700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25c4105ee0 0x7f25c419d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c8a57700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c419db10 con 0x7f25c41035c0 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c8a57700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c419dc50 con 0x7f25c4105ee0 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c8a57700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25c419dde0 con 0x7f25c4100ca0 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 0x7f25c41964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 0x7f25c41964d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49350/0 (socket says 172.21.15.93:49350) 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c1d9b700 1 -- 172.21.15.93:0/766165256 learned_addr learned my addr 172.21.15.93:0/766165256 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c259c700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25c4100ca0 0x7f25c4195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c2d9d700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25c4105ee0 0x7f25c419d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c1d9b700 1 -- 172.21.15.93:0/766165256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25c4100ca0 msgr2=0x7f25c4195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c1d9b700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25c4100ca0 0x7f25c4195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c1d9b700 1 -- 172.21.15.93:0/766165256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25c4105ee0 msgr2=0x7f25c419d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c1d9b700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25c4105ee0 0x7f25c419d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c1d9b700 1 -- 172.21.15.93:0/766165256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b80095f0 con 0x7f25c41035c0 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c2d9d700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25c4105ee0 0x7f25c419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:48.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c1d9b700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 0x7f25c41964d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f25b8000c00 tx=0x7f25b800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:48.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25b37fe700 1 -- 172.21.15.93:0/766165256 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b801d070 con 0x7f25c41035c0 2023-12-14T18:24:48.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25b37fe700 1 -- 172.21.15.93:0/766165256 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f25b800fb50 con 0x7f25c41035c0 2023-12-14T18:24:48.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25c410bba0 con 0x7f25c41035c0 2023-12-14T18:24:48.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25b37fe700 1 -- 172.21.15.93:0/766165256 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25b8017650 con 0x7f25c41035c0 2023-12-14T18:24:48.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.062+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25c410ba10 con 0x7f25c41035c0 2023-12-14T18:24:48.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.063+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25c40fc2b0 con 0x7f25c41035c0 2023-12-14T18:24:48.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.064+0000 7f25b37fe700 1 -- 172.21.15.93:0/766165256 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f25b800fcc0 con 0x7f25c41035c0 2023-12-14T18:24:48.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.065+0000 7f25b37fe700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25ac0a1150 0x7f25ac0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.065+0000 7f25c259c700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25ac0a1150 0x7f25ac0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.065+0000 7f25b37fe700 1 -- 172.21.15.93:0/766165256 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f25b80cde50 con 0x7f25c41035c0 2023-12-14T18:24:48.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.066+0000 7f25c259c700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25ac0a1150 0x7f25ac0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005950 tx=0x7f25b400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:48.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.069+0000 7f25b37fe700 1 -- 172.21.15.93:0/766165256 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f25b8097440 con 0x7f25c41035c0 2023-12-14T18:24:48.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:47 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1152670241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:48.245 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:47 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1152670241' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:48.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.257+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f25c4061c40 con 0x7f25c41035c0 2023-12-14T18:24:48.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.258+0000 7f25b37fe700 1 -- 172.21.15.93:0/766165256 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7f25b809aa60 con 0x7f25c41035c0 2023-12-14T18:24:48.259 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25ac0a1150 msgr2=0x7f25ac0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25ac0a1150 0x7f25ac0a3600 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f25b4005950 tx=0x7f25b400a300 comp rx=0 tx=0).stop 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 msgr2=0x7f25c41964d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 0x7f25c41964d0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f25b8000c00 tx=0x7f25b800f710 comp rx=0 tx=0).stop 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 shutdown_connections 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25ac0a1150 0x7f25ac0a3600 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f25c4105ee0 0x7f25c419d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f25c4100ca0 0x7f25c4195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 --2- 172.21.15.93:0/766165256 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f25c41035c0 0x7f25c41964d0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 >> 172.21.15.93:0/766165256 conn(0x7f25c40fa740 msgr2=0x7f25c4101900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 shutdown_connections 2023-12-14T18:24:48.262 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.260+0000 7f25c8a57700 1 -- 172.21.15.93:0/766165256 wait complete. 2023-12-14T18:24:48.273 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:24:48.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 -- 172.21.15.93:0/209808626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100ce0 msgr2=0x7f5af01075a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 --2- 172.21.15.93:0/209808626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100ce0 0x7f5af01075a0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f5ae800b0a0 tx=0x7f5ae800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 -- 172.21.15.93:0/209808626 shutdown_connections 2023-12-14T18:24:48.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 --2- 172.21.15.93:0/209808626 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100ce0 0x7f5af01075a0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 --2- 172.21.15.93:0/209808626 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af00ff160 0x7f5af00ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 --2- 172.21.15.93:0/209808626 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af0100350 0x7f5af01007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 -- 172.21.15.93:0/209808626 >> 172.21.15.93:0/209808626 conn(0x7f5af00fa780 msgr2=0x7f5af00fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:48.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 -- 172.21.15.93:0/209808626 shutdown_connections 2023-12-14T18:24:48.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 -- 172.21.15.93:0/209808626 wait complete. 2023-12-14T18:24:48.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.397+0000 7f5af4efa700 1 Processor -- start 2023-12-14T18:24:48.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5af4efa700 1 -- start start 2023-12-14T18:24:48.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5af4efa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af00ff160 0x7f5af0196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5af4efa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100350 0x7f5af01965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5af4efa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0100ce0 0x7f5af019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5af4efa700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af019dc10 con 0x7f5af0100350 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5af4efa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af019dd50 con 0x7f5af0100ce0 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5af4efa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5af019dee0 con 0x7f5af00ff160 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aedd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100350 0x7f5af01965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aedd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100350 0x7f5af01965d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49364/0 (socket says 172.21.15.93:49364) 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aedd9b700 1 -- 172.21.15.93:0/1963648940 learned_addr learned my addr 172.21.15.93:0/1963648940 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aee59c700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af00ff160 0x7f5af0196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aeed9d700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0100ce0 0x7f5af019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aedd9b700 1 -- 172.21.15.93:0/1963648940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af00ff160 msgr2=0x7f5af0196090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aedd9b700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af00ff160 0x7f5af0196090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aedd9b700 1 -- 172.21.15.93:0/1963648940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0100ce0 msgr2=0x7f5af019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aedd9b700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0100ce0 0x7f5af019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aedd9b700 1 -- 172.21.15.93:0/1963648940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae8009d00 con 0x7f5af0100350 2023-12-14T18:24:48.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aeed9d700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0100ce0 0x7f5af019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:24:48.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5aedd9b700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100350 0x7f5af01965d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d950 tx=0x7f5ae000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:48.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5adf7fe700 1 -- 172.21.15.93:0/1963648940 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0009940 con 0x7f5af0100350 2023-12-14T18:24:48.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5adf7fe700 1 -- 172.21.15.93:0/1963648940 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ae0010460 con 0x7f5af0100350 2023-12-14T18:24:48.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5af010ab30 con 0x7f5af0100350 2023-12-14T18:24:48.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.398+0000 7f5adf7fe700 1 -- 172.21.15.93:0/1963648940 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ae000f620 con 0x7f5af0100350 2023-12-14T18:24:48.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.399+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5af010a9a0 con 0x7f5af0100350 2023-12-14T18:24:48.403 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.399+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5af019e5d0 con 0x7f5af0100350 2023-12-14T18:24:48.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.400+0000 7f5adf7fe700 1 -- 172.21.15.93:0/1963648940 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5ae00108d0 con 0x7f5af0100350 2023-12-14T18:24:48.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.401+0000 7f5adf7fe700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5ad80a11d0 0x7f5ad80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.401+0000 7f5aee59c700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5ad80a11d0 0x7f5ad80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.401+0000 7f5adf7fe700 1 -- 172.21.15.93:0/1963648940 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5ae00cc9d0 con 0x7f5af0100350 2023-12-14T18:24:48.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.402+0000 7f5aee59c700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5ad80a11d0 0x7f5ad80a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5af01001b0 tx=0x7f5ae400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:48.407 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.405+0000 7f5adf7fe700 1 -- 172.21.15.93:0/1963648940 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5ae0095ed0 con 0x7f5af0100350 2023-12-14T18:24:48.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.597+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5af019eb80 con 0x7f5af0100350 2023-12-14T18:24:48.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.597+0000 7f5adf7fe700 1 -- 172.21.15.93:0/1963648940 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ae00160a0 con 0x7f5af0100350 2023-12-14T18:24:48.599 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-12-14T18:24:48.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.599+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5ad80a11d0 msgr2=0x7f5ad80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.599+0000 7f5af4efa700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5ad80a11d0 0x7f5ad80a3680 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5af01001b0 tx=0x7f5ae400b410 comp rx=0 tx=0).stop 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.599+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100350 msgr2=0x7f5af01965d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.599+0000 7f5af4efa700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100350 0x7f5af01965d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5ae000d950 tx=0x7f5ae000dc60 comp rx=0 tx=0).stop 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.600+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 shutdown_connections 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.600+0000 7f5af4efa700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5ad80a11d0 0x7f5ad80a3680 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.600+0000 7f5af4efa700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5af0100ce0 0x7f5af019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.600+0000 7f5af4efa700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5af00ff160 0x7f5af0196090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.600+0000 7f5af4efa700 1 --2- 172.21.15.93:0/1963648940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5af0100350 0x7f5af01965d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.600+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 >> 172.21.15.93:0/1963648940 conn(0x7f5af00fa780 msgr2=0x7f5af0068da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.600+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 shutdown_connections 2023-12-14T18:24:48.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.600+0000 7f5af4efa700 1 -- 172.21.15.93:0/1963648940 wait complete. 2023-12-14T18:24:48.614 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:24:48.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.737+0000 7fe3ff222700 1 -- 172.21.15.93:0/2290627546 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 msgr2=0x7fe3f81018f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.737+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2290627546 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 0x7fe3f81018f0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe3e4009990 tx=0x7fe3e4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.737+0000 7fe3ff222700 1 -- 172.21.15.93:0/2290627546 shutdown_connections 2023-12-14T18:24:48.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.737+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2290627546 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f8101e30 0x7fe3f81086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.737+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2290627546 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f80fe4d0 0x7fe3f80fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.741 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.737+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2290627546 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 0x7fe3f81018f0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.737+0000 7fe3ff222700 1 -- 172.21.15.93:0/2290627546 >> 172.21.15.93:0/2290627546 conn(0x7fe3f80f9ed0 msgr2=0x7fe3f80fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 -- 172.21.15.93:0/2290627546 shutdown_connections 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 -- 172.21.15.93:0/2290627546 wait complete. 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 Processor -- start 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 -- start start 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f80fe4d0 0x7fe3f8195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 0x7fe3f81964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f8101e30 0x7fe3f819d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f819db00 con 0x7fe3f81014a0 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f819dc40 con 0x7fe3f8101e30 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3ff222700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3f819ddd0 con 0x7fe3f80fe4d0 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 0x7fe3f81964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3f7fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 0x7fe3f81964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49382/0 (socket says 172.21.15.93:49382) 2023-12-14T18:24:48.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3f7fff700 1 -- 172.21.15.93:0/2780914445 learned_addr learned my addr 172.21.15.93:0/2780914445 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.738+0000 7fe3fcfbe700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f80fe4d0 0x7fe3f8195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3fd7bf700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f8101e30 0x7fe3f819d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3f7fff700 1 -- 172.21.15.93:0/2780914445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f80fe4d0 msgr2=0x7fe3f8195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3f7fff700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f80fe4d0 0x7fe3f8195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3f7fff700 1 -- 172.21.15.93:0/2780914445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f8101e30 msgr2=0x7fe3f819d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3f7fff700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f8101e30 0x7fe3f819d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3f7fff700 1 -- 172.21.15.93:0/2780914445 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3e40095f0 con 0x7fe3f81014a0 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3f7fff700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 0x7fe3f81964c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40098e0 tx=0x7fe3e400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3f5ffb700 1 -- 172.21.15.93:0/2780914445 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e401d070 con 0x7fe3f81014a0 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3f5ffb700 1 -- 172.21.15.93:0/2780914445 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe3e4022470 con 0x7fe3f81014a0 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3f810bb70 con 0x7fe3f81014a0 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3f5ffb700 1 -- 172.21.15.93:0/2780914445 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3e4017650 con 0x7fe3f81014a0 2023-12-14T18:24:48.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.739+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe3f810b9e0 con 0x7fe3f81014a0 2023-12-14T18:24:48.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.740+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe3f8107230 con 0x7fe3f81014a0 2023-12-14T18:24:48.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.741+0000 7fe3f5ffb700 1 -- 172.21.15.93:0/2780914445 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe3e40225e0 con 0x7fe3f81014a0 2023-12-14T18:24:48.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.742+0000 7fe3f5ffb700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe3e80a1150 0x7fe3e80a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:48.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.742+0000 7fe3fcfbe700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe3e80a1150 0x7fe3e80a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:48.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.742+0000 7fe3f5ffb700 1 -- 172.21.15.93:0/2780914445 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe3e40cdff0 con 0x7fe3f81014a0 2023-12-14T18:24:48.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.742+0000 7fe3fcfbe700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe3e80a1150 0x7fe3e80a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8067bd0 tx=0x7fe3ec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:48.747 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.745+0000 7fe3f5ffb700 1 -- 172.21.15.93:0/2780914445 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe3e40975c0 con 0x7fe3f81014a0 2023-12-14T18:24:48.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.935+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fe3f8061700 con 0x7fe3f81014a0 2023-12-14T18:24:48.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.935+0000 7fe3f5ffb700 1 -- 172.21.15.93:0/2780914445 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7fe3e409abe0 con 0x7fe3f81014a0 2023-12-14T18:24:48.937 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:24:48.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe3e80a1150 msgr2=0x7fe3e80a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe3e80a1150 0x7fe3e80a3600 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe3f8067bd0 tx=0x7fe3ec00b410 comp rx=0 tx=0).stop 2023-12-14T18:24:48.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 msgr2=0x7fe3f81964c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:48.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 0x7fe3f81964c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe3e40098e0 tx=0x7fe3e400f690 comp rx=0 tx=0).stop 2023-12-14T18:24:48.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 shutdown_connections 2023-12-14T18:24:48.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe3e80a1150 0x7fe3e80a3600 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe3f8101e30 0x7fe3f819d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe3f80fe4d0 0x7fe3f8195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 --2- 172.21.15.93:0/2780914445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe3f81014a0 0x7fe3f81964c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:48.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 >> 172.21.15.93:0/2780914445 conn(0x7fe3f80f9ed0 msgr2=0x7fe3f80fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:48.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 shutdown_connections 2023-12-14T18:24:48.940 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:48.937+0000 7fe3ff222700 1 -- 172.21.15.93:0/2780914445 wait complete. 2023-12-14T18:24:48.950 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:48 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3320332509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:48.951 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:48 smithi093 ceph-mon[42070]: pgmap v230: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 4 op/s 2023-12-14T18:24:48.951 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:48 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/766165256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:24:48.951 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:48 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1963648940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:24:48.952 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-14T18:24:49.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.072+0000 7f78ac095700 1 -- 172.21.15.93:0/862162341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a40ff120 msgr2=0x7f78a40ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.072+0000 7f78ac095700 1 --2- 172.21.15.93:0/862162341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a40ff120 0x7f78a40ff530 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f7894009990 tx=0x7f7894009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.072+0000 7f78ac095700 1 -- 172.21.15.93:0/862162341 shutdown_connections 2023-12-14T18:24:49.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.072+0000 7f78ac095700 1 --2- 172.21.15.93:0/862162341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a40ff120 0x7f78a40ff530 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.072+0000 7f78ac095700 1 --2- 172.21.15.93:0/862162341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a4100ca0 0x7f78a4107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.072+0000 7f78ac095700 1 --2- 172.21.15.93:0/862162341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100310 0x7f78a4100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.072+0000 7f78ac095700 1 -- 172.21.15.93:0/862162341 >> 172.21.15.93:0/862162341 conn(0x7f78a40fa740 msgr2=0x7f78a40fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.072+0000 7f78ac095700 1 -- 172.21.15.93:0/862162341 shutdown_connections 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.072+0000 7f78ac095700 1 -- 172.21.15.93:0/862162341 wait complete. 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78ac095700 1 Processor -- start 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78ac095700 1 -- start start 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78ac095700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a40ff120 0x7f78a4194ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78ac095700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100310 0x7f78a4195400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78ac095700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100ca0 0x7f78a419c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78ac095700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419ca40 con 0x7f78a4100310 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78ac095700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419cb80 con 0x7f78a40ff120 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78ac095700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419cd10 con 0x7f78a4100ca0 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78a9630700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100310 0x7f78a4195400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78a9630700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100310 0x7f78a4195400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49402/0 (socket says 172.21.15.93:49402) 2023-12-14T18:24:49.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78a9630700 1 -- 172.21.15.93:0/3627968576 learned_addr learned my addr 172.21.15.93:0/3627968576 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.073+0000 7f78a9e31700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a40ff120 0x7f78a4194ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78aa632700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100ca0 0x7f78a419c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78a9630700 1 -- 172.21.15.93:0/3627968576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100ca0 msgr2=0x7f78a419c470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78a9630700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100ca0 0x7f78a419c470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78a9630700 1 -- 172.21.15.93:0/3627968576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a40ff120 msgr2=0x7f78a4194ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78a9630700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a40ff120 0x7f78a4194ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78a9630700 1 -- 172.21.15.93:0/3627968576 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78940095f0 con 0x7f78a4100310 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78aa632700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100ca0 0x7f78a419c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78a9630700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100310 0x7f78a4195400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f78a000c920 tx=0x7f78a000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f789affd700 1 -- 172.21.15.93:0/3627968576 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0007b00 con 0x7f78a4100310 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f789affd700 1 -- 172.21.15.93:0/3627968576 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f78a000ce70 con 0x7f78a4100310 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a410aa90 con 0x7f78a4100310 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f789affd700 1 -- 172.21.15.93:0/3627968576 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78a0005460 con 0x7f78a4100310 2023-12-14T18:24:49.078 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.074+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a410a900 con 0x7f78a4100310 2023-12-14T18:24:49.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.075+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a41060a0 con 0x7f78a4100310 2023-12-14T18:24:49.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.077+0000 7f789affd700 1 -- 172.21.15.93:0/3627968576 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f78a00055c0 con 0x7f78a4100310 2023-12-14T18:24:49.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.077+0000 7f789affd700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78900a1150 0x7f78900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.077+0000 7f789affd700 1 -- 172.21.15.93:0/3627968576 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f78a00cca80 con 0x7f78a4100310 2023-12-14T18:24:49.079 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.077+0000 7f78a9e31700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78900a1150 0x7f78900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.080 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.078+0000 7f78a9e31700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78900a1150 0x7f78900a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f789400b5c0 tx=0x7f7894005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:49.082 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.080+0000 7f789affd700 1 -- 172.21.15.93:0/3627968576 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f78a0095fe0 con 0x7f78a4100310 2023-12-14T18:24:49.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:48 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3320332509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:49.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:48 smithi097 ceph-mon[43486]: pgmap v230: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 4 op/s 2023-12-14T18:24:49.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:48 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/766165256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:24:49.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:48 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1963648940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:24:49.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:48 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3320332509' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:49.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:48 smithi160 ceph-mon[43840]: pgmap v230: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 4 op/s 2023-12-14T18:24:49.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:48 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/766165256' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:24:49.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:48 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1963648940' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:24:49.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.282+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f78a4061820 con 0x7f78a4100310 2023-12-14T18:24:49.284 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.283+0000 7f789affd700 1 -- 172.21.15.93:0/3627968576 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) v1 ==== 81+0+590 (secure 0 0 0) 0x7f78a001d020 con 0x7f78a4100310 2023-12-14T18:24:49.284 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/2647964405 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:49.284 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3363004642 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:49.285 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2583563521 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:49.285 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1226958737 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:49.285 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/1555415520 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:49.285 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/216269584 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:49.285 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/1555415520 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:49.285 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/2647964405 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:49.285 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/630684095 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:49.285 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2376464512 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:49.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78900a1150 msgr2=0x7f78900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78900a1150 0x7f78900a3600 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f789400b5c0 tx=0x7f7894005960 comp rx=0 tx=0).stop 2023-12-14T18:24:49.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100310 msgr2=0x7f78a4195400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100310 0x7f78a4195400 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f78a000c920 tx=0x7f78a000cc30 comp rx=0 tx=0).stop 2023-12-14T18:24:49.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 shutdown_connections 2023-12-14T18:24:49.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78900a1150 0x7f78900a3600 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100ca0 0x7f78a419c470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a40ff120 0x7f78a4194ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 --2- 172.21.15.93:0/3627968576 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100310 0x7f78a4195400 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 >> 172.21.15.93:0/3627968576 conn(0x7f78a40fa740 msgr2=0x7f78a40fca20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:49.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 shutdown_connections 2023-12-14T18:24:49.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.285+0000 7f78ac095700 1 -- 172.21.15.93:0/3627968576 wait complete. 2023-12-14T18:24:49.288 INFO:teuthology.orchestra.run.smithi093.stderr:listed 10 entries 2023-12-14T18:24:49.299 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:24:49.299 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:24:49.317 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:24:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.477+0000 7f5803a06700 1 -- 172.21.15.93:0/966603424 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc0ff110 msgr2=0x7f57fc0ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.477+0000 7f5803a06700 1 --2- 172.21.15.93:0/966603424 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc0ff110 0x7f57fc0ff520 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f57f0009990 tx=0x7f57f0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.477+0000 7f5803a06700 1 -- 172.21.15.93:0/966603424 shutdown_connections 2023-12-14T18:24:49.479 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.477+0000 7f5803a06700 1 --2- 172.21.15.93:0/966603424 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc0ff110 0x7f57fc0ff520 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.477+0000 7f5803a06700 1 --2- 172.21.15.93:0/966603424 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57fc100c90 0x7f57fc107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.477+0000 7f5803a06700 1 --2- 172.21.15.93:0/966603424 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57fc100300 0x7f57fc100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.477+0000 7f5803a06700 1 -- 172.21.15.93:0/966603424 >> 172.21.15.93:0/966603424 conn(0x7f57fc0fa6f0 msgr2=0x7f57fc0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.477+0000 7f5803a06700 1 -- 172.21.15.93:0/966603424 shutdown_connections 2023-12-14T18:24:49.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.477+0000 7f5803a06700 1 -- 172.21.15.93:0/966603424 wait complete. 2023-12-14T18:24:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5803a06700 1 Processor -- start 2023-12-14T18:24:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5803a06700 1 -- start start 2023-12-14T18:24:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5803a06700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57fc0ff110 0x7f57fc195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5803a06700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57fc100300 0x7f57fc1964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5803a06700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc100c90 0x7f57fc19d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5803a06700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19db30 con 0x7f57fc100c90 2023-12-14T18:24:49.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5803a06700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19dc70 con 0x7f57fc100300 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5803a06700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57fc19de00 con 0x7f57fc0ff110 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5801fa3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc100c90 0x7f57fc19d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5801fa3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc100c90 0x7f57fc19d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49408/0 (socket says 172.21.15.93:49408) 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5801fa3700 1 -- 172.21.15.93:0/2298428332 learned_addr learned my addr 172.21.15.93:0/2298428332 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f58017a2700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57fc0ff110 0x7f57fc195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.478+0000 7f5800fa1700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57fc100300 0x7f57fc1964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f5801fa3700 1 -- 172.21.15.93:0/2298428332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57fc0ff110 msgr2=0x7f57fc195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f5801fa3700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57fc0ff110 0x7f57fc195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f5801fa3700 1 -- 172.21.15.93:0/2298428332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57fc100300 msgr2=0x7f57fc1964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f5801fa3700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57fc100300 0x7f57fc1964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f5801fa3700 1 -- 172.21.15.93:0/2298428332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57f00095f0 con 0x7f57fc100c90 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f5801fa3700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc100c90 0x7f57fc19d560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f57f4007ee0 tx=0x7f57f400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f57ee7fc700 1 -- 172.21.15.93:0/2298428332 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4017070 con 0x7f57fc100c90 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f57ee7fc700 1 -- 172.21.15.93:0/2298428332 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57f400f040 con 0x7f57fc100c90 2023-12-14T18:24:49.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57fc10aa70 con 0x7f57fc100c90 2023-12-14T18:24:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f57ee7fc700 1 -- 172.21.15.93:0/2298428332 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57f4005580 con 0x7f57fc100c90 2023-12-14T18:24:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.479+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57fc10a8e0 con 0x7f57fc100c90 2023-12-14T18:24:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.480+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57fc19e4f0 con 0x7f57fc100c90 2023-12-14T18:24:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.481+0000 7f57ee7fc700 1 -- 172.21.15.93:0/2298428332 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f57f400db20 con 0x7f57fc100c90 2023-12-14T18:24:49.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.481+0000 7f57ee7fc700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57e80a11d0 0x7f57e80a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.482+0000 7f58017a2700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57e80a11d0 0x7f57e80a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.482+0000 7f57ee7fc700 1 -- 172.21.15.93:0/2298428332 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f57f40cc840 con 0x7f57fc100c90 2023-12-14T18:24:49.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.482+0000 7f58017a2700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57e80a11d0 0x7f57e80a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b5c0 tx=0x7f57f0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:49.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.485+0000 7f57ee7fc700 1 -- 172.21.15.93:0/2298428332 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f57f4095da0 con 0x7f57fc100c90 2023-12-14T18:24:49.681 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.679+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f57fc19eaa0 con 0x7f57fc100c90 2023-12-14T18:24:49.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.681+0000 7f57ee7fc700 1 -- 172.21.15.93:0/2298428332 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7f57f40993c0 con 0x7f57fc100c90 2023-12-14T18:24:49.683 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:49.683 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57e80a11d0 msgr2=0x7f57e80a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57e80a11d0 0x7f57e80a3680 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f57f000b5c0 tx=0x7f57f0005960 comp rx=0 tx=0).stop 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc100c90 msgr2=0x7f57fc19d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc100c90 0x7f57fc19d560 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f57f4007ee0 tx=0x7f57f400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 shutdown_connections 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57e80a11d0 0x7f57e80a3680 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57fc100c90 0x7f57fc19d560 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57fc0ff110 0x7f57fc195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 --2- 172.21.15.93:0/2298428332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57fc100300 0x7f57fc1964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.683+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 >> 172.21.15.93:0/2298428332 conn(0x7f57fc0fa6f0 msgr2=0x7f57fc067dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:49.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.684+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 shutdown_connections 2023-12-14T18:24:49.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.684+0000 7f5803a06700 1 -- 172.21.15.93:0/2298428332 wait complete. 2023-12-14T18:24:49.698 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:24:34.407045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:24:35.728548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:24:38.796189+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:24:49.698 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:24:49.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.824+0000 7fd755a4b700 1 -- 172.21.15.93:0/3653030765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750100d10 msgr2=0x7fd7501030f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.824+0000 7fd755a4b700 1 --2- 172.21.15.93:0/3653030765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750100d10 0x7fd7501030f0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fd740009990 tx=0x7fd740009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.824+0000 7fd755a4b700 1 -- 172.21.15.93:0/3653030765 shutdown_connections 2023-12-14T18:24:49.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.824+0000 7fd755a4b700 1 --2- 172.21.15.93:0/3653030765 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750100d10 0x7fd7501030f0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.824+0000 7fd755a4b700 1 --2- 172.21.15.93:0/3653030765 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750105f50 0x7fd750108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.824+0000 7fd755a4b700 1 --2- 172.21.15.93:0/3653030765 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd750103630 0x7fd750105a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.824+0000 7fd755a4b700 1 -- 172.21.15.93:0/3653030765 >> 172.21.15.93:0/3653030765 conn(0x7fd7500fa750 msgr2=0x7fd7500fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.824+0000 7fd755a4b700 1 -- 172.21.15.93:0/3653030765 shutdown_connections 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd755a4b700 1 -- 172.21.15.93:0/3653030765 wait complete. 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd755a4b700 1 Processor -- start 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd755a4b700 1 -- start start 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd755a4b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750100d10 0x7fd750195fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd755a4b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd750103630 0x7fd7501964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd755a4b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750105f50 0x7fd75019d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd755a4b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75019db30 con 0x7fd750105f50 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd755a4b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75019dc70 con 0x7fd750100d10 2023-12-14T18:24:49.830 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd755a4b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75019de00 con 0x7fd750103630 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750105f50 0x7fd75019d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd74f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750105f50 0x7fd75019d560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49412/0 (socket says 172.21.15.93:49412) 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.825+0000 7fd74f7fe700 1 -- 172.21.15.93:0/2333259358 learned_addr learned my addr 172.21.15.93:0/2333259358 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd74effd700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750100d10 0x7fd750195fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd74e7fc700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd750103630 0x7fd7501964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd74f7fe700 1 -- 172.21.15.93:0/2333259358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd750103630 msgr2=0x7fd7501964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd74f7fe700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd750103630 0x7fd7501964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd74f7fe700 1 -- 172.21.15.93:0/2333259358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750100d10 msgr2=0x7fd750195fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd74f7fe700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750100d10 0x7fd750195fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd74f7fe700 1 -- 172.21.15.93:0/2333259358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7400095f0 con 0x7fd750105f50 2023-12-14T18:24:49.831 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd74e7fc700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd750103630 0x7fd7501964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd74f7fe700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750105f50 0x7fd75019d560 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd748007ee0 tx=0x7fd74800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd754a49700 1 -- 172.21.15.93:0/2333259358 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd74800ddd0 con 0x7fd750105f50 2023-12-14T18:24:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75010bb90 con 0x7fd750105f50 2023-12-14T18:24:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd754a49700 1 -- 172.21.15.93:0/2333259358 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd74800f040 con 0x7fd750105f50 2023-12-14T18:24:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd754a49700 1 -- 172.21.15.93:0/2333259358 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7480048e0 con 0x7fd750105f50 2023-12-14T18:24:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.826+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75010ba00 con 0x7fd750105f50 2023-12-14T18:24:49.832 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.827+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7500fc2c0 con 0x7fd750105f50 2023-12-14T18:24:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.828+0000 7fd754a49700 1 -- 172.21.15.93:0/2333259358 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd748003a00 con 0x7fd750105f50 2023-12-14T18:24:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.829+0000 7fd754a49700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7380a1150 0x7fd7380a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.829+0000 7fd754a49700 1 -- 172.21.15.93:0/2333259358 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd7480ccc50 con 0x7fd750105f50 2023-12-14T18:24:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.829+0000 7fd74effd700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7380a1150 0x7fd7380a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:49.833 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.829+0000 7fd74effd700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7380a1150 0x7fd7380a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b5c0 tx=0x7fd740005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:49.834 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:49.832+0000 7fd754a49700 1 -- 172.21.15.93:0/2333259358 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd748096150 con 0x7fd750105f50 2023-12-14T18:24:50.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.022+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd750067e40 con 0x7fd750105f50 2023-12-14T18:24:50.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.023+0000 7fd754a49700 1 -- 172.21.15.93:0/2333259358 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v84) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7480230a0 con 0x7fd750105f50 2023-12-14T18:24:50.025 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:24:50.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.024+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7380a1150 msgr2=0x7fd7380a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.024+0000 7fd755a4b700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7380a1150 0x7fd7380a3600 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd74000b5c0 tx=0x7fd740005960 comp rx=0 tx=0).stop 2023-12-14T18:24:50.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.024+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750105f50 msgr2=0x7fd75019d560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.024+0000 7fd755a4b700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750105f50 0x7fd75019d560 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fd748007ee0 tx=0x7fd74800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.025+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 shutdown_connections 2023-12-14T18:24:50.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.025+0000 7fd755a4b700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd7380a1150 0x7fd7380a3600 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.025+0000 7fd755a4b700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd750105f50 0x7fd75019d560 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.025+0000 7fd755a4b700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd750100d10 0x7fd750195fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.025+0000 7fd755a4b700 1 --2- 172.21.15.93:0/2333259358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd750103630 0x7fd7501964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.025+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 >> 172.21.15.93:0/2333259358 conn(0x7fd7500fa750 msgr2=0x7fd750101a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:50.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.025+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 shutdown_connections 2023-12-14T18:24:50.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.025+0000 7fd755a4b700 1 -- 172.21.15.93:0/2333259358 wait complete. 2023-12-14T18:24:50.037 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:49 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2780914445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:24:50.038 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:49 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3627968576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:24:50.038 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:49 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2298428332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:50.039 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:24:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:49 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2780914445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:24:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:49 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3627968576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:24:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:49 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2298428332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:50.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 -- 172.21.15.93:0/1160463936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 msgr2=0x7fd9681059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.166 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 --2- 172.21.15.93:0/1160463936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 0x7fd9681059c0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fd964009990 tx=0x7fd964009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 -- 172.21.15.93:0/1160463936 shutdown_connections 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 --2- 172.21.15.93:0/1160463936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd968105f00 0x7fd968108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 --2- 172.21.15.93:0/1160463936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd968100cc0 0x7fd9681030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 --2- 172.21.15.93:0/1160463936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 0x7fd9681059c0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 -- 172.21.15.93:0/1160463936 >> 172.21.15.93:0/1160463936 conn(0x7fd9680fa740 msgr2=0x7fd9680fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 -- 172.21.15.93:0/1160463936 shutdown_connections 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 -- 172.21.15.93:0/1160463936 wait complete. 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 Processor -- start 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.162+0000 7fd96f964700 1 -- start start 2023-12-14T18:24:50.167 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96f964700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd968100cc0 0x7fd968195e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96f964700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 0x7fd968196350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96f964700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd968105f00 0x7fd96819d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96f964700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96819d990 con 0x7fd9681035e0 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96f964700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96819dad0 con 0x7fd968105f00 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96f964700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd96819dc60 con 0x7fd968100cc0 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96ceff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 0x7fd968196350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96ceff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 0x7fd968196350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49436/0 (socket says 172.21.15.93:49436) 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96ceff700 1 -- 172.21.15.93:0/1028579832 learned_addr learned my addr 172.21.15.93:0/1028579832 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96d700700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd968100cc0 0x7fd968195e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.168 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96df01700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd968105f00 0x7fd96819d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96ceff700 1 -- 172.21.15.93:0/1028579832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd968100cc0 msgr2=0x7fd968195e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96ceff700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd968100cc0 0x7fd968195e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96ceff700 1 -- 172.21.15.93:0/1028579832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd968105f00 msgr2=0x7fd96819d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96ceff700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd968105f00 0x7fd96819d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96ceff700 1 -- 172.21.15.93:0/1028579832 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9640095f0 con 0x7fd9681035e0 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96df01700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd968105f00 0x7fd96819d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96d700700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd968100cc0 0x7fd968195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96ceff700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 0x7fd968196350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd964000c00 tx=0x7fd96400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd95e7fc700 1 -- 172.21.15.93:0/1028579832 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd96401d070 con 0x7fd9681035e0 2023-12-14T18:24:50.169 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd95e7fc700 1 -- 172.21.15.93:0/1028579832 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd96400fc30 con 0x7fd9681035e0 2023-12-14T18:24:50.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd96810bbc0 con 0x7fd9681035e0 2023-12-14T18:24:50.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd95e7fc700 1 -- 172.21.15.93:0/1028579832 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd964017770 con 0x7fd9681035e0 2023-12-14T18:24:50.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.163+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd96810ba30 con 0x7fd9681035e0 2023-12-14T18:24:50.170 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.164+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9680fc2b0 con 0x7fd9681035e0 2023-12-14T18:24:50.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.166+0000 7fd95e7fc700 1 -- 172.21.15.93:0/1028579832 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd964022a30 con 0x7fd9681035e0 2023-12-14T18:24:50.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.166+0000 7fd95e7fc700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd9540a1150 0x7fd9540a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.167+0000 7fd95e7fc700 1 -- 172.21.15.93:0/1028579832 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd9640ce180 con 0x7fd9681035e0 2023-12-14T18:24:50.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.167+0000 7fd96d700700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd9540a1150 0x7fd9540a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.171 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.167+0000 7fd96d700700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd9540a1150 0x7fd9540a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd958005fd0 tx=0x7fd958005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:50.172 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.169+0000 7fd95e7fc700 1 -- 172.21.15.93:0/1028579832 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd964097710 con 0x7fd9681035e0 2023-12-14T18:24:50.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:49 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2780914445' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:24:50.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:49 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3627968576' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:24:50.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:49 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2298428332' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:50.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.360+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd968061980 con 0x7fd9681035e0 2023-12-14T18:24:50.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.361+0000 7fd95e7fc700 1 -- 172.21.15.93:0/1028579832 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v84) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd96409ad30 con 0x7fd9681035e0 2023-12-14T18:24:50.363 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:24:50.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd9540a1150 msgr2=0x7fd9540a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd9540a1150 0x7fd9540a3600 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd958005fd0 tx=0x7fd958005e30 comp rx=0 tx=0).stop 2023-12-14T18:24:50.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 msgr2=0x7fd968196350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 0x7fd968196350 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fd964000c00 tx=0x7fd96400f790 comp rx=0 tx=0).stop 2023-12-14T18:24:50.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 shutdown_connections 2023-12-14T18:24:50.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd9540a1150 0x7fd9540a3600 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd968105f00 0x7fd96819d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd968100cc0 0x7fd968195e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 --2- 172.21.15.93:0/1028579832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd9681035e0 0x7fd968196350 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 >> 172.21.15.93:0/1028579832 conn(0x7fd9680fa740 msgr2=0x7fd968101920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:50.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 shutdown_connections 2023-12-14T18:24:50.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.363+0000 7fd96f964700 1 -- 172.21.15.93:0/1028579832 wait complete. 2023-12-14T18:24:50.376 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:24:50.505 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.500+0000 7fbd7ded0700 1 -- 172.21.15.93:0/3568917491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd78100330 msgr2=0x7fbd78100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.500+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/3568917491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd78100330 0x7fbd78100780 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fbd60009990 tx=0x7fbd60009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.500+0000 7fbd7ded0700 1 -- 172.21.15.93:0/3568917491 shutdown_connections 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.500+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/3568917491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd78100cc0 0x7fbd78107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.500+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/3568917491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd780ff140 0x7fbd780ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.500+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/3568917491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd78100330 0x7fbd78100780 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.500+0000 7fbd7ded0700 1 -- 172.21.15.93:0/3568917491 >> 172.21.15.93:0/3568917491 conn(0x7fbd780fa740 msgr2=0x7fbd780fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 -- 172.21.15.93:0/3568917491 shutdown_connections 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 -- 172.21.15.93:0/3568917491 wait complete. 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 Processor -- start 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 -- start start 2023-12-14T18:24:50.506 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd780ff140 0x7fbd78195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd78100330 0x7fbd78196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78100cc0 0x7fbd7819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7819db50 con 0x7fbd780ff140 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7819dc90 con 0x7fbd78100330 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd7ded0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd7819de20 con 0x7fbd78100cc0 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd780ff140 0x7fbd78195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd777fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd780ff140 0x7fbd78195fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49452/0 (socket says 172.21.15.93:49452) 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.501+0000 7fbd777fe700 1 -- 172.21.15.93:0/1079474845 learned_addr learned my addr 172.21.15.93:0/1079474845 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd77fff700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78100cc0 0x7fbd7819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd6ffff700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd78100330 0x7fbd78196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd777fe700 1 -- 172.21.15.93:0/1079474845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78100cc0 msgr2=0x7fbd7819d580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd777fe700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78100cc0 0x7fbd7819d580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd777fe700 1 -- 172.21.15.93:0/1079474845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd78100330 msgr2=0x7fbd78196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd777fe700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd78100330 0x7fbd78196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd777fe700 1 -- 172.21.15.93:0/1079474845 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd600095f0 con 0x7fbd780ff140 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd777fe700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd780ff140 0x7fbd78195fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800e950 tx=0x7fbd6800ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd757fa700 1 -- 172.21.15.93:0/1079474845 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68009770 con 0x7fbd780ff140 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd757fa700 1 -- 172.21.15.93:0/1079474845 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd680044f0 con 0x7fbd780ff140 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd757fa700 1 -- 172.21.15.93:0/1079474845 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd68003e00 con 0x7fbd780ff140 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd7810aac0 con 0x7fbd780ff140 2023-12-14T18:24:50.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.502+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd7810a930 con 0x7fbd780ff140 2023-12-14T18:24:50.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.503+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd7819e520 con 0x7fbd780ff140 2023-12-14T18:24:50.509 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.504+0000 7fbd757fa700 1 -- 172.21.15.93:0/1079474845 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbd68003f80 con 0x7fbd780ff140 2023-12-14T18:24:50.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.504+0000 7fbd757fa700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbd640a11d0 0x7fbd640a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.505+0000 7fbd6ffff700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbd640a11d0 0x7fbd640a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.505+0000 7fbd757fa700 1 -- 172.21.15.93:0/1079474845 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fbd68014070 con 0x7fbd780ff140 2023-12-14T18:24:50.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.505+0000 7fbd6ffff700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbd640a11d0 0x7fbd640a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd60006010 tx=0x7fbd6000b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:50.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.509+0000 7fbd757fa700 1 -- 172.21.15.93:0/1079474845 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbd68096440 con 0x7fbd780ff140 2023-12-14T18:24:50.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.696+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fbd7819ead0 con 0x7fbd780ff140 2023-12-14T18:24:50.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.696+0000 7fbd757fa700 1 -- 172.21.15.93:0/1079474845 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v84) v1 ==== 100+0+11 (secure 0 0 0) 0x7fbd68099a60 con 0x7fbd780ff140 2023-12-14T18:24:50.699 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-12-14T18:24:50.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbd640a11d0 msgr2=0x7fbd640a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbd640a11d0 0x7fbd640a3680 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fbd60006010 tx=0x7fbd6000b260 comp rx=0 tx=0).stop 2023-12-14T18:24:50.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd780ff140 msgr2=0x7fbd78195fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd780ff140 0x7fbd78195fd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fbd6800e950 tx=0x7fbd6800ed10 comp rx=0 tx=0).stop 2023-12-14T18:24:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 shutdown_connections 2023-12-14T18:24:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbd640a11d0 0x7fbd640a3680 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd780ff140 0x7fbd78195fd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd78100cc0 0x7fbd7819d580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 --2- 172.21.15.93:0/1079474845 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd78100330 0x7fbd78196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 >> 172.21.15.93:0/1079474845 conn(0x7fbd780fa740 msgr2=0x7fbd78068e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 shutdown_connections 2023-12-14T18:24:50.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.699+0000 7fbd7ded0700 1 -- 172.21.15.93:0/1079474845 wait complete. 2023-12-14T18:24:50.713 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:24:50.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.837+0000 7f656cfdd700 1 -- 172.21.15.93:0/348288326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680ff3e0 msgr2=0x7f65680ff850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.837+0000 7f656cfdd700 1 --2- 172.21.15.93:0/348288326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680ff3e0 0x7f65680ff850 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f655c009990 tx=0x7f655c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 -- 172.21.15.93:0/348288326 shutdown_connections 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 --2- 172.21.15.93:0/348288326 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65680ffd90 0x7f6568100200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 --2- 172.21.15.93:0/348288326 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65680fea90 0x7f65680feea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 --2- 172.21.15.93:0/348288326 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680ff3e0 0x7f65680ff850 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 -- 172.21.15.93:0/348288326 >> 172.21.15.93:0/348288326 conn(0x7f65680fa740 msgr2=0x7f65680fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 -- 172.21.15.93:0/348288326 shutdown_connections 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 -- 172.21.15.93:0/348288326 wait complete. 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 Processor -- start 2023-12-14T18:24:50.843 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 -- start start 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680fea90 0x7f6568198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.838+0000 7f656cfdd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65680ff3e0 0x7f6568198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656cfdd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65680ffd90 0x7f656819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656cfdd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656819fd90 con 0x7f65680fea90 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656cfdd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f656819fed0 con 0x7f65680ff3e0 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656cfdd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65681a0060 con 0x7f65680ffd90 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680fea90 0x7f6568198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656659c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680fea90 0x7f6568198210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49474/0 (socket says 172.21.15.93:49474) 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656659c700 1 -- 172.21.15.93:0/2636992984 learned_addr learned my addr 172.21.15.93:0/2636992984 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:50.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f6566d9d700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65680ffd90 0x7f656819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f6565d9b700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65680ff3e0 0x7f6568198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656659c700 1 -- 172.21.15.93:0/2636992984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65680ffd90 msgr2=0x7f656819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656659c700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65680ffd90 0x7f656819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656659c700 1 -- 172.21.15.93:0/2636992984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65680ff3e0 msgr2=0x7f6568198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:50.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656659c700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65680ff3e0 0x7f6568198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:50.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656659c700 1 -- 172.21.15.93:0/2636992984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f655c0095f0 con 0x7f65680fea90 2023-12-14T18:24:50.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656659c700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680fea90 0x7f6568198210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f655800e9a0 tx=0x7f655800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:50.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f65577fe700 1 -- 172.21.15.93:0/2636992984 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558009790 con 0x7f65680fea90 2023-12-14T18:24:50.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f65577fe700 1 -- 172.21.15.93:0/2636992984 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f655800cbd0 con 0x7f65680fea90 2023-12-14T18:24:50.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f656810dd70 con 0x7f65680fea90 2023-12-14T18:24:50.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f65577fe700 1 -- 172.21.15.93:0/2636992984 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6558018680 con 0x7f65680fea90 2023-12-14T18:24:50.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.839+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f656810dbe0 con 0x7f65680fea90 2023-12-14T18:24:50.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.840+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6568109870 con 0x7f65680fea90 2023-12-14T18:24:50.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.842+0000 7f65577fe700 1 -- 172.21.15.93:0/2636992984 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f655800cd40 con 0x7f65680fea90 2023-12-14T18:24:50.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.842+0000 7f65577fe700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f65500a1150 0x7f65500a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:50.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.843+0000 7f6565d9b700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f65500a1150 0x7f65500a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:50.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.843+0000 7f65577fe700 1 -- 172.21.15.93:0/2636992984 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6558014070 con 0x7f65680fea90 2023-12-14T18:24:50.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.843+0000 7f6565d9b700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f65500a1150 0x7f65500a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f655c006010 tx=0x7f655c00b260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:50.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:50.846+0000 7f65577fe700 1 -- 172.21.15.93:0/2636992984 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6558096590 con 0x7f65680fea90 2023-12-14T18:24:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.034+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f65680679f0 con 0x7f65680fea90 2023-12-14T18:24:51.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.034+0000 7f65577fe700 1 -- 172.21.15.93:0/2636992984 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v84) v1 ==== 103+0+11 (secure 0 0 0) 0x7f6558094e60 con 0x7f65680fea90 2023-12-14T18:24:51.036 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:24:51.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f65500a1150 msgr2=0x7f65500a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:51.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f65500a1150 0x7f65500a3600 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f655c006010 tx=0x7f655c00b260 comp rx=0 tx=0).stop 2023-12-14T18:24:51.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680fea90 msgr2=0x7f6568198210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:51.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680fea90 0x7f6568198210 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f655800e9a0 tx=0x7f655800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 shutdown_connections 2023-12-14T18:24:51.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f65500a1150 0x7f65500a3600 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f65680fea90 0x7f6568198210 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f65680ffd90 0x7f656819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 --2- 172.21.15.93:0/2636992984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f65680ff3e0 0x7f6568198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 >> 172.21.15.93:0/2636992984 conn(0x7f65680fa740 msgr2=0x7f65680fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 shutdown_connections 2023-12-14T18:24:51.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.036+0000 7f656cfdd700 1 -- 172.21.15.93:0/2636992984 wait complete. 2023-12-14T18:24:51.049 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:50 smithi093 ceph-mon[42070]: pgmap v231: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 437 B/s wr, 3 op/s 2023-12-14T18:24:51.049 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:50 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2333259358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:51.049 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:50 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1028579832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:24:51.049 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:50 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1079474845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:24:51.051 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-14T18:24:51.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:50 smithi097 ceph-mon[43486]: pgmap v231: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 437 B/s wr, 3 op/s 2023-12-14T18:24:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:50 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2333259358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:50 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1028579832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:24:51.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:50 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1079474845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:24:51.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 -- 172.21.15.93:0/3923240514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 msgr2=0x7f4838100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:51.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 --2- 172.21.15.93:0/3923240514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 0x7f4838100760 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4834009990 tx=0x7f4834009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 -- 172.21.15.93:0/3923240514 shutdown_connections 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 --2- 172.21.15.93:0/3923240514 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838100ca0 0x7f4838107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 --2- 172.21.15.93:0/3923240514 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48380ff120 0x7f48380ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 --2- 172.21.15.93:0/3923240514 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 0x7f4838100760 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 -- 172.21.15.93:0/3923240514 >> 172.21.15.93:0/3923240514 conn(0x7f48380fa740 msgr2=0x7f48380fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 -- 172.21.15.93:0/3923240514 shutdown_connections 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 -- 172.21.15.93:0/3923240514 wait complete. 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 Processor -- start 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 -- start start 2023-12-14T18:24:51.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.172+0000 7f483f535700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48380ff120 0x7f4838068940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483f535700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 0x7f483819acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483f535700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838100ca0 0x7f483819d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483f535700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483819dad0 con 0x7f4838100310 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483f535700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483819dc40 con 0x7f4838100ca0 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483f535700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f483819de00 con 0x7f48380ff120 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483cad0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 0x7f483819acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483cad0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 0x7f483819acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49490/0 (socket says 172.21.15.93:49490) 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483cad0700 1 -- 172.21.15.93:0/3451342264 learned_addr learned my addr 172.21.15.93:0/3451342264 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483dad2700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838100ca0 0x7f483819d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483d2d1700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48380ff120 0x7f4838068940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:51.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483cad0700 1 -- 172.21.15.93:0/3451342264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48380ff120 msgr2=0x7f4838068940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483cad0700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48380ff120 0x7f4838068940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483cad0700 1 -- 172.21.15.93:0/3451342264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838100ca0 msgr2=0x7f483819d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483cad0700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838100ca0 0x7f483819d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483cad0700 1 -- 172.21.15.93:0/3451342264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48340095f0 con 0x7f4838100310 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483dad2700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838100ca0 0x7f483819d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483cad0700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 0x7f483819acf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4834004760 tx=0x7f4834009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f482e7fc700 1 -- 172.21.15.93:0/3451342264 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483401d070 con 0x7f4838100310 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f483810aa80 con 0x7f4838100310 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f482e7fc700 1 -- 172.21.15.93:0/3451342264 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4834004200 con 0x7f4838100310 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f482e7fc700 1 -- 172.21.15.93:0/3451342264 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48340049f0 con 0x7f4838100310 2023-12-14T18:24:51.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.173+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f483810a8f0 con 0x7f4838100310 2023-12-14T18:24:51.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.174+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4838106050 con 0x7f4838100310 2023-12-14T18:24:51.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.175+0000 7f482e7fc700 1 -- 172.21.15.93:0/3451342264 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f4834003d40 con 0x7f4838100310 2023-12-14T18:24:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.176+0000 7f482e7fc700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f48240a1150 0x7f48240a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.176+0000 7f482e7fc700 1 -- 172.21.15.93:0/3451342264 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f48340cde70 con 0x7f4838100310 2023-12-14T18:24:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.176+0000 7f483d2d1700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f48240a1150 0x7f48240a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.176+0000 7f483d2d1700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f48240a1150 0x7f48240a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4828005950 tx=0x7f48280058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:51.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.179+0000 7f482e7fc700 1 -- 172.21.15.93:0/3451342264 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4834097460 con 0x7f4838100310 2023-12-14T18:24:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:50 smithi160 ceph-mon[43840]: pgmap v231: 129 pgs: 129 active+clean; 579 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 437 B/s wr, 3 op/s 2023-12-14T18:24:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:50 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2333259358' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:24:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:50 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1028579832' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:24:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:50 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1079474845' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:24:51.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.382+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f4838061900 con 0x7f4838100310 2023-12-14T18:24:51.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.382+0000 7f482e7fc700 1 -- 172.21.15.93:0/3451342264 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v84) v1 ==== 81+0+590 (secure 0 0 0) 0x7f483409aa80 con 0x7f4838100310 2023-12-14T18:24:51.384 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/2647964405 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:51.384 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3363004642 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:51.384 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2583563521 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:51.384 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1226958737 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:51.384 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/1555415520 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:51.384 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/216269584 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:51.385 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/1555415520 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:51.385 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/2647964405 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:51.385 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/630684095 2023-12-15T18:19:15.054604+0000 2023-12-14T18:24:51.385 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2376464512 2023-12-15T18:19:35.884432+0000 2023-12-14T18:24:51.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f48240a1150 msgr2=0x7f48240a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:51.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f48240a1150 0x7f48240a3600 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f4828005950 tx=0x7f48280058e0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 msgr2=0x7f483819acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:51.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 0x7f483819acf0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f4834004760 tx=0x7f4834009f90 comp rx=0 tx=0).stop 2023-12-14T18:24:51.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 shutdown_connections 2023-12-14T18:24:51.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f48240a1150 0x7f48240a3600 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4838100ca0 0x7f483819d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f48380ff120 0x7f4838068940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 --2- 172.21.15.93:0/3451342264 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4838100310 0x7f483819acf0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:51.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 >> 172.21.15.93:0/3451342264 conn(0x7f48380fa740 msgr2=0x7f48380fc9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:51.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 shutdown_connections 2023-12-14T18:24:51.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:51.384+0000 7f483f535700 1 -- 172.21.15.93:0/3451342264 wait complete. 2023-12-14T18:24:51.387 INFO:teuthology.orchestra.run.smithi093.stderr:listed 10 entries 2023-12-14T18:24:51.398 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi093.front.sepia.ceph.com... 2023-12-14T18:24:51.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:51.398 DEBUG:teuthology.orchestra.run.smithi093:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2023-12-14T18:24:51.438 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi093.stderr:ceph-fuse[78737]: fuse finished with error 0 and tester_r 0 2023-12-14T18:24:51.439 INFO:teuthology.orchestra.run:waiting for 300 2023-12-14T18:24:52.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:51 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2636992984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:24:52.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:51 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3451342264' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:24:52.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:51 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2636992984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:24:52.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:51 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3451342264' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:24:52.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:51 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2636992984' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:24:52.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:51 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3451342264' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:24:53.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:52 smithi093 ceph-mon[42070]: pgmap v232: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 561 B/s wr, 0 op/s 2023-12-14T18:24:53.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:52 smithi097 ceph-mon[43486]: pgmap v232: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 561 B/s wr, 0 op/s 2023-12-14T18:24:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:52 smithi160 ceph-mon[43840]: pgmap v232: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 561 B/s wr, 0 op/s 2023-12-14T18:24:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:55 smithi093 ceph-mon[42070]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:24:55.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:55 smithi097 ceph-mon[43486]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:24:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:55 smithi160 ceph-mon[43840]: pgmap v233: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:24:57.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:56 smithi093 ceph-mon[42070]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:24:57.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:56 smithi097 ceph-mon[43486]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:24:57.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:56 smithi160 ceph-mon[43840]: pgmap v234: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2023-12-14T18:24:57.541 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:24:57.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:24:57.542 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2023-12-14T18:24:57.576 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:24:57.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 -- 172.21.15.93:0/506641033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 msgr2=0x7f2ca8103150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:57.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 --2- 172.21.15.93:0/506641033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 0x7f2ca8103150 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c009990 tx=0x7f2c9c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 -- 172.21.15.93:0/506641033 shutdown_connections 2023-12-14T18:24:57.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 --2- 172.21.15.93:0/506641033 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 0x7f2ca8103150 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 --2- 172.21.15.93:0/506641033 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ca8105fb0 0x7f2ca8108390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 --2- 172.21.15.93:0/506641033 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ca8103690 0x7f2ca8105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 -- 172.21.15.93:0/506641033 >> 172.21.15.93:0/506641033 conn(0x7f2ca80fa750 msgr2=0x7f2ca80fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:57.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 -- 172.21.15.93:0/506641033 shutdown_connections 2023-12-14T18:24:57.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 -- 172.21.15.93:0/506641033 wait complete. 2023-12-14T18:24:57.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 Processor -- start 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.750+0000 7f2cad95f700 1 -- start start 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cad95f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 0x7f2ca81006e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cad95f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ca8103690 0x7f2ca80fed20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cad95f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ca8105fb0 0x7f2ca80ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cad95f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80ffa00 con 0x7f2ca8100d70 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cad95f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca80ffb70 con 0x7f2ca8105fb0 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cad95f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca81a0080 con 0x7f2ca8103690 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 0x7f2ca81006e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 0x7f2ca81006e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37882/0 (socket says 172.21.15.93:37882) 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca6ffd700 1 -- 172.21.15.93:0/2623331977 learned_addr learned my addr 172.21.15.93:0/2623331977 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:57.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca77fe700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ca8105fb0 0x7f2ca80ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca67fc700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ca8103690 0x7f2ca80fed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca6ffd700 1 -- 172.21.15.93:0/2623331977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ca8103690 msgr2=0x7f2ca80fed20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca6ffd700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ca8103690 0x7f2ca80fed20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca6ffd700 1 -- 172.21.15.93:0/2623331977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ca8105fb0 msgr2=0x7f2ca80ff260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca6ffd700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ca8105fb0 0x7f2ca80ff260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca6ffd700 1 -- 172.21.15.93:0/2623331977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c9c0095f0 con 0x7f2ca8100d70 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca67fc700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ca8103690 0x7f2ca80fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2ca6ffd700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 0x7f2ca81006e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0095c0 tx=0x7f2c9c00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cac95d700 1 -- 172.21.15.93:0/2623331977 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c01d070 con 0x7f2ca8100d70 2023-12-14T18:24:57.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca810bc20 con 0x7f2ca8100d70 2023-12-14T18:24:57.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cac95d700 1 -- 172.21.15.93:0/2623331977 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2c9c003e20 con 0x7f2ca8100d70 2023-12-14T18:24:57.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cac95d700 1 -- 172.21.15.93:0/2623331977 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c9c0173f0 con 0x7f2ca8100d70 2023-12-14T18:24:57.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.751+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca810ba90 con 0x7f2ca8100d70 2023-12-14T18:24:57.758 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.753+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca8067e40 con 0x7f2ca8100d70 2023-12-14T18:24:57.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.754+0000 7f2cac95d700 1 -- 172.21.15.93:0/2623331977 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2c9c003960 con 0x7f2ca8100d70 2023-12-14T18:24:57.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.754+0000 7f2cac95d700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c900a1150 0x7f2c900a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:57.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.754+0000 7f2cac95d700 1 -- 172.21.15.93:0/2623331977 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2c9c0cdca0 con 0x7f2ca8100d70 2023-12-14T18:24:57.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.754+0000 7f2ca67fc700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c900a1150 0x7f2c900a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:57.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.755+0000 7f2ca67fc700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c900a1150 0x7f2c900a3600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c98005fd0 tx=0x7f2c98005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:57.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.758+0000 7f2cac95d700 1 -- 172.21.15.93:0/2623331977 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2c9c097290 con 0x7f2ca8100d70 2023-12-14T18:24:57.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.951+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2ca81a11b0 con 0x7f2ca8100d70 2023-12-14T18:24:57.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.952+0000 7f2cac95d700 1 -- 172.21.15.93:0/2623331977 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7f2c9c09a8b0 con 0x7f2ca8100d70 2023-12-14T18:24:57.954 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:57.955 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.954+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c900a1150 msgr2=0x7f2c900a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.954+0000 7f2cad95f700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c900a1150 0x7f2c900a3600 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2c98005fd0 tx=0x7f2c98005ee0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 msgr2=0x7f2ca81006e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 0x7f2ca81006e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2c9c0095c0 tx=0x7f2c9c00f690 comp rx=0 tx=0).stop 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 shutdown_connections 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2c900a1150 0x7f2c900a3600 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ca8100d70 0x7f2ca81006e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ca8105fb0 0x7f2ca80ff260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 --2- 172.21.15.93:0/2623331977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ca8103690 0x7f2ca80fed20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 >> 172.21.15.93:0/2623331977 conn(0x7f2ca80fa750 msgr2=0x7f2ca8102220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:57.957 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 shutdown_connections 2023-12-14T18:24:57.958 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:57.955+0000 7f2cad95f700 1 -- 172.21.15.93:0/2623331977 wait complete. 2023-12-14T18:24:57.969 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.089+0000 7ff06b0d0700 1 -- 172.21.15.93:0/1170594457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064100230 msgr2=0x7ff064100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.089+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/1170594457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064100230 0x7ff064100640 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7ff058009990 tx=0x7ff058009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.089+0000 7ff06b0d0700 1 -- 172.21.15.93:0/1170594457 shutdown_connections 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.089+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/1170594457 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064100230 0x7ff064100640 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.089+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/1170594457 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff064101db0 0x7ff064108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.089+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/1170594457 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff064101420 0x7ff064101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.089+0000 7ff06b0d0700 1 -- 172.21.15.93:0/1170594457 >> 172.21.15.93:0/1170594457 conn(0x7ff0640fb810 msgr2=0x7ff0640fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.089+0000 7ff06b0d0700 1 -- 172.21.15.93:0/1170594457 shutdown_connections 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.089+0000 7ff06b0d0700 1 -- 172.21.15.93:0/1170594457 wait complete. 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff06b0d0700 1 Processor -- start 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff06b0d0700 1 -- start start 2023-12-14T18:24:58.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff06b0d0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff064100230 0x7ff064196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff06b0d0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064101420 0x7ff064196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff06b0d0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff064101db0 0x7ff06419d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff06b0d0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06419db90 con 0x7ff064101420 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff06b0d0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06419dcd0 con 0x7ff064101db0 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff06b0d0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06419de60 con 0x7ff064100230 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff063fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064101420 0x7ff064196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff063fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064101420 0x7ff064196550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37908/0 (socket says 172.21.15.93:37908) 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff063fff700 1 -- 172.21.15.93:0/183823855 learned_addr learned my addr 172.21.15.93:0/183823855 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff068e6c700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff064100230 0x7ff064196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff06966d700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff064101db0 0x7ff06419d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff063fff700 1 -- 172.21.15.93:0/183823855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff064100230 msgr2=0x7ff064196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff063fff700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff064100230 0x7ff064196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff063fff700 1 -- 172.21.15.93:0/183823855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff064101db0 msgr2=0x7ff06419d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff063fff700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff064101db0 0x7ff06419d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.090+0000 7ff063fff700 1 -- 172.21.15.93:0/183823855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0580095f0 con 0x7ff064101420 2023-12-14T18:24:58.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.091+0000 7ff063fff700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064101420 0x7ff064196550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c970 tx=0x7ff05000cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:58.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.091+0000 7ff061ffb700 1 -- 172.21.15.93:0/183823855 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050007aa0 con 0x7ff064101420 2023-12-14T18:24:58.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.091+0000 7ff061ffb700 1 -- 172.21.15.93:0/183823855 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff050004cf0 con 0x7ff064101420 2023-12-14T18:24:58.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.091+0000 7ff061ffb700 1 -- 172.21.15.93:0/183823855 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050005600 con 0x7ff064101420 2023-12-14T18:24:58.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.091+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff06410bbc0 con 0x7ff064101420 2023-12-14T18:24:58.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.091+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff06410ba30 con 0x7ff064101420 2023-12-14T18:24:58.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.092+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06419e560 con 0x7ff064101420 2023-12-14T18:24:58.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.093+0000 7ff061ffb700 1 -- 172.21.15.93:0/183823855 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ff050004750 con 0x7ff064101420 2023-12-14T18:24:58.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.093+0000 7ff061ffb700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ff0540a11d0 0x7ff0540a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:58.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.093+0000 7ff061ffb700 1 -- 172.21.15.93:0/183823855 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7ff050013070 con 0x7ff064101420 2023-12-14T18:24:58.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.094+0000 7ff068e6c700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ff0540a11d0 0x7ff0540a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:58.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.094+0000 7ff068e6c700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ff0540a11d0 0x7ff0540a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff058005b40 tx=0x7ff058005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:58.099 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.098+0000 7ff061ffb700 1 -- 172.21.15.93:0/183823855 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ff050095e70 con 0x7ff064101420 2023-12-14T18:24:58.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.288+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff064107160 con 0x7ff064101420 2023-12-14T18:24:58.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.289+0000 7ff061ffb700 1 -- 172.21.15.93:0/183823855 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7ff050004e60 con 0x7ff064101420 2023-12-14T18:24:58.291 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:24:58.292 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.291+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ff0540a11d0 msgr2=0x7ff0540a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.291+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ff0540a11d0 0x7ff0540a3680 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff058005b40 tx=0x7ff058005ab0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.291+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064101420 msgr2=0x7ff064196550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.291+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064101420 0x7ff064196550 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c970 tx=0x7ff05000cd30 comp rx=0 tx=0).stop 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.292+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 shutdown_connections 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.292+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ff0540a11d0 0x7ff0540a3680 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.292+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff064101db0 0x7ff06419d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.292+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff064100230 0x7ff064196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.292+0000 7ff06b0d0700 1 --2- 172.21.15.93:0/183823855 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff064101420 0x7ff064196550 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.292+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 >> 172.21.15.93:0/183823855 conn(0x7ff0640fb810 msgr2=0x7ff0640fdad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.292+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 shutdown_connections 2023-12-14T18:24:58.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.292+0000 7ff06b0d0700 1 -- 172.21.15.93:0/183823855 wait complete. 2023-12-14T18:24:58.304 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:58 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2623331977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:58.306 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-14T18:24:58.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:58 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2623331977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 -- 172.21.15.93:0/3453592566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c74100350 msgr2=0x7f1c741007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 --2- 172.21.15.93:0/3453592566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c74100350 0x7f1c741007a0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1c64009990 tx=0x7f1c64009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 -- 172.21.15.93:0/3453592566 shutdown_connections 2023-12-14T18:24:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 --2- 172.21.15.93:0/3453592566 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c74100ce0 0x7f1c741075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.433 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 --2- 172.21.15.93:0/3453592566 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c740ff160 0x7f1c740ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 --2- 172.21.15.93:0/3453592566 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c74100350 0x7f1c741007a0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 -- 172.21.15.93:0/3453592566 >> 172.21.15.93:0/3453592566 conn(0x7f1c740fa780 msgr2=0x7f1c740fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 -- 172.21.15.93:0/3453592566 shutdown_connections 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 -- 172.21.15.93:0/3453592566 wait complete. 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 Processor -- start 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.429+0000 7f1c79571700 1 -- start start 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c79571700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c740ff160 0x7f1c74194e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c79571700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c74100350 0x7f1c74195380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c79571700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c74100ce0 0x7f1c7419c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c79571700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7419c9c0 con 0x7f1c740ff160 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c79571700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7419cb00 con 0x7f1c74100350 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c79571700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7419cc90 con 0x7f1c74100ce0 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c72ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c740ff160 0x7f1c74194e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:58.434 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c72ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c740ff160 0x7f1c74194e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37932/0 (socket says 172.21.15.93:37932) 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c72ffd700 1 -- 172.21.15.93:0/1167192030 learned_addr learned my addr 172.21.15.93:0/1167192030 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c727fc700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c74100350 0x7f1c74195380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c737fe700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c74100ce0 0x7f1c7419c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c72ffd700 1 -- 172.21.15.93:0/1167192030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c74100ce0 msgr2=0x7f1c7419c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c72ffd700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c74100ce0 0x7f1c7419c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c72ffd700 1 -- 172.21.15.93:0/1167192030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c74100350 msgr2=0x7f1c74195380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c72ffd700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c74100350 0x7f1c74195380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c72ffd700 1 -- 172.21.15.93:0/1167192030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c640095f0 con 0x7f1c740ff160 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c72ffd700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c740ff160 0x7f1c74194e40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800ea80 tx=0x7f1c6800ed90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c5bfff700 1 -- 172.21.15.93:0/1167192030 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c68013070 con 0x7f1c740ff160 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c7410aac0 con 0x7f1c740ff160 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c5bfff700 1 -- 172.21.15.93:0/1167192030 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1c6800be80 con 0x7f1c740ff160 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.430+0000 7f1c5bfff700 1 -- 172.21.15.93:0/1167192030 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c680039c0 con 0x7f1c740ff160 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.431+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c7410a930 con 0x7f1c740ff160 2023-12-14T18:24:58.435 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.431+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c74067160 con 0x7f1c740ff160 2023-12-14T18:24:58.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.432+0000 7f1c5bfff700 1 -- 172.21.15.93:0/1167192030 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1c68004120 con 0x7f1c740ff160 2023-12-14T18:24:58.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.433+0000 7f1c5bfff700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1c5c0a11d0 0x7f1c5c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:58.436 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.433+0000 7f1c727fc700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1c5c0a11d0 0x7f1c5c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:58.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.433+0000 7f1c5bfff700 1 -- 172.21.15.93:0/1167192030 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f1c680ce120 con 0x7f1c740ff160 2023-12-14T18:24:58.437 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.434+0000 7f1c727fc700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1c5c0a11d0 0x7f1c5c0a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400b2e0 tx=0x7f1c64005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:58.439 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.437+0000 7f1c5bfff700 1 -- 172.21.15.93:0/1167192030 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1c68097620 con 0x7f1c740ff160 2023-12-14T18:24:58.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:58 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2623331977' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:58.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:58.672+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f1c74106510 con 0x7f1c740ff160 2023-12-14T18:24:59.014 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.012+0000 7f1c5bfff700 1 -- 172.21.15.93:0/1167192030 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) v1 ==== 118+0+0 (secure 0 0 0) 0x7f1c6809ac40 con 0x7f1c740ff160 2023-12-14T18:24:59.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.014+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1c5c0a11d0 msgr2=0x7f1c5c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:59.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.014+0000 7f1c79571700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1c5c0a11d0 0x7f1c5c0a3680 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f1c6400b2e0 tx=0x7f1c64005960 comp rx=0 tx=0).stop 2023-12-14T18:24:59.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.014+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c740ff160 msgr2=0x7f1c74194e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:59.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.014+0000 7f1c79571700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c740ff160 0x7f1c74194e40 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1c6800ea80 tx=0x7f1c6800ed90 comp rx=0 tx=0).stop 2023-12-14T18:24:59.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.014+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 shutdown_connections 2023-12-14T18:24:59.017 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.014+0000 7f1c79571700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1c5c0a11d0 0x7f1c5c0a3680 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.014+0000 7f1c79571700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1c740ff160 0x7f1c74194e40 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.014+0000 7f1c79571700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1c74100ce0 0x7f1c7419c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.015+0000 7f1c79571700 1 --2- 172.21.15.93:0/1167192030 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1c74100350 0x7f1c74195380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.015+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 >> 172.21.15.93:0/1167192030 conn(0x7f1c740fa780 msgr2=0x7f1c74103790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:59.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.015+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 shutdown_connections 2023-12-14T18:24:59.018 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.015+0000 7f1c79571700 1 -- 172.21.15.93:0/1167192030 wait complete. 2023-12-14T18:24:59.030 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 3, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2023-12-14T18:24:59.030 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 3 2023-12-14T18:24:59.159 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.154+0000 7f78aaf7f700 1 -- 172.21.15.93:0/496377736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100d50 msgr2=0x7f78a4103130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.154+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/496377736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100d50 0x7f78a4103130 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f7898009990 tx=0x7f7898009ca0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.154+0000 7f78aaf7f700 1 -- 172.21.15.93:0/496377736 shutdown_connections 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.154+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/496377736 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4100d50 0x7f78a4103130 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.154+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/496377736 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4105f90 0x7f78a4108370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.154+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/496377736 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a4103670 0x7f78a4105a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.154+0000 7f78aaf7f700 1 -- 172.21.15.93:0/496377736 >> 172.21.15.93:0/496377736 conn(0x7f78a40fa750 msgr2=0x7f78a40fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.154+0000 7f78aaf7f700 1 -- 172.21.15.93:0/496377736 shutdown_connections 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.154+0000 7f78aaf7f700 1 -- 172.21.15.93:0/496377736 wait complete. 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78aaf7f700 1 Processor -- start 2023-12-14T18:24:59.160 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78aaf7f700 1 -- start start 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78aaf7f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100d50 0x7f78a41006c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78aaf7f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4103670 0x7f78a40fed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78aaf7f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a4105f90 0x7f78a40ff240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78aaf7f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40ff9e0 con 0x7f78a4103670 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78aaf7f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a40ffb50 con 0x7f78a4105f90 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78aaf7f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78a419ff60 con 0x7f78a4100d50 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78a3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4103670 0x7f78a40fed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78a3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4103670 0x7f78a40fed00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37944/0 (socket says 172.21.15.93:37944) 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78a3fff700 1 -- 172.21.15.93:0/1351863517 learned_addr learned my addr 172.21.15.93:0/1351863517 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:24:59.161 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.155+0000 7f78a8d1b700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100d50 0x7f78a41006c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a951c700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a4105f90 0x7f78a40ff240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a3fff700 1 -- 172.21.15.93:0/1351863517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100d50 msgr2=0x7f78a41006c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a3fff700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100d50 0x7f78a41006c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a3fff700 1 -- 172.21.15.93:0/1351863517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a4105f90 msgr2=0x7f78a40ff240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a3fff700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a4105f90 0x7f78a40ff240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a3fff700 1 -- 172.21.15.93:0/1351863517 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78980095f0 con 0x7f78a4103670 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a3fff700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4103670 0x7f78a40fed00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f789000c9b0 tx=0x7f789000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a1ffb700 1 -- 172.21.15.93:0/1351863517 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890007b00 con 0x7f78a4103670 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a1ffb700 1 -- 172.21.15.93:0/1351863517 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7890007c60 con 0x7f78a4103670 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78a410bb90 con 0x7f78a4103670 2023-12-14T18:24:59.162 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78a1ffb700 1 -- 172.21.15.93:0/1351863517 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7890018660 con 0x7f78a4103670 2023-12-14T18:24:59.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.156+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78a410ba00 con 0x7f78a4103670 2023-12-14T18:24:59.163 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.157+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78a40fc2c0 con 0x7f78a4103670 2023-12-14T18:24:59.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.158+0000 7f78a1ffb700 1 -- 172.21.15.93:0/1351863517 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f789000f450 con 0x7f78a4103670 2023-12-14T18:24:59.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.159+0000 7f78a1ffb700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78940a1150 0x7f78940a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:24:59.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.159+0000 7f78a8d1b700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78940a1150 0x7f78940a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:24:59.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.159+0000 7f78a1ffb700 1 -- 172.21.15.93:0/1351863517 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f78900ccb50 con 0x7f78a4103670 2023-12-14T18:24:59.164 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.159+0000 7f78a8d1b700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78940a1150 0x7f78940a3600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f789800b5c0 tx=0x7f7898005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:24:59.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.163+0000 7f78a1ffb700 1 -- 172.21.15.93:0/1351863517 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7890096050 con 0x7f78a4103670 2023-12-14T18:24:59.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:59 smithi093 ceph-mon[42070]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-12-14T18:24:59.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:59 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/183823855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:59.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:24:59 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1167192030' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-14T18:24:59.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:59 smithi097 ceph-mon[43486]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-12-14T18:24:59.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:59 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/183823855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:59.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:24:59 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1167192030' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-14T18:24:59.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:24:59.395+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"} v 0) v1 -- 0x7f78a4061be0 con 0x7f78a4103670 2023-12-14T18:24:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:59 smithi160 ceph-mon[43840]: pgmap v235: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-12-14T18:24:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:59 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/183823855' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:24:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:24:59 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1167192030' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2023-12-14T18:25:00.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.023+0000 7f78a1ffb700 1 -- 172.21.15.93:0/1351863517 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]=0 v19) v1 ==== 105+0+0 (secure 0 0 0) 0x7f7890099670 con 0x7f78a4103670 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.025+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78940a1150 msgr2=0x7f78940a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.025+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78940a1150 0x7f78940a3600 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f789800b5c0 tx=0x7f7898005fd0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.025+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4103670 msgr2=0x7f78a40fed00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.025+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4103670 0x7f78a40fed00 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f789000c9b0 tx=0x7f789000ccc0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.026+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 shutdown_connections 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.026+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f78940a1150 0x7f78940a3600 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.026+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f78a4105f90 0x7f78a40ff240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.026+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f78a4100d50 0x7f78a41006c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.026+0000 7f78aaf7f700 1 --2- 172.21.15.93:0/1351863517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f78a4103670 0x7f78a40fed00 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.026+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 >> 172.21.15.93:0/1351863517 conn(0x7f78a40fa750 msgr2=0x7f78a41019d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.026+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 shutdown_connections 2023-12-14T18:25:00.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.026+0000 7f78aaf7f700 1 -- 172.21.15.93:0/1351863517 wait complete. 2023-12-14T18:25:00.045 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2023-12-14T18:25:00.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.171+0000 7f2591591700 1 -- 172.21.15.93:0/494745143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c101e30 msgr2=0x7f258c1086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.171+0000 7f2591591700 1 --2- 172.21.15.93:0/494745143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c101e30 0x7f258c1086f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f258400b0a0 tx=0x7f258400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.172+0000 7f2591591700 1 -- 172.21.15.93:0/494745143 shutdown_connections 2023-12-14T18:25:00.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.172+0000 7f2591591700 1 --2- 172.21.15.93:0/494745143 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c101e30 0x7f258c1086f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.172+0000 7f2591591700 1 --2- 172.21.15.93:0/494745143 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c0fe4d0 0x7f258c0fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.172+0000 7f2591591700 1 --2- 172.21.15.93:0/494745143 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258c1014a0 0x7f258c1018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.172+0000 7f2591591700 1 -- 172.21.15.93:0/494745143 >> 172.21.15.93:0/494745143 conn(0x7f258c0f9ed0 msgr2=0x7f258c0fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:00.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.172+0000 7f2591591700 1 -- 172.21.15.93:0/494745143 shutdown_connections 2023-12-14T18:25:00.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.172+0000 7f2591591700 1 -- 172.21.15.93:0/494745143 wait complete. 2023-12-14T18:25:00.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.172+0000 7f2591591700 1 Processor -- start 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2591591700 1 -- start start 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2591591700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258c0fe4d0 0x7f258c06f460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2591591700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c1014a0 0x7f258c06f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2591591700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c101e30 0x7f258c1a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2591591700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c06ff70 con 0x7f258c1014a0 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2591591700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c0700e0 con 0x7f258c101e30 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2591591700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f258c1a6700 con 0x7f258c0fe4d0 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2583fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c1014a0 0x7f258c06f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2583fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c1014a0 0x7f258c06f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37974/0 (socket says 172.21.15.93:37974) 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2583fff700 1 -- 172.21.15.93:0/1390089481 learned_addr learned my addr 172.21.15.93:0/1390089481 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f258affd700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258c0fe4d0 0x7f258c06f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f258b7fe700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c101e30 0x7f258c1a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2583fff700 1 -- 172.21.15.93:0/1390089481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258c0fe4d0 msgr2=0x7f258c06f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2583fff700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258c0fe4d0 0x7f258c06f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2583fff700 1 -- 172.21.15.93:0/1390089481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c101e30 msgr2=0x7f258c1a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2583fff700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c101e30 0x7f258c1a6170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2583fff700 1 -- 172.21.15.93:0/1390089481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f257c009520 con 0x7f258c1014a0 2023-12-14T18:25:00.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2583fff700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c1014a0 0x7f258c06f9a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f257400e9a0 tx=0x7f257400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:00.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.173+0000 7f2588ff9700 1 -- 172.21.15.93:0/1390089481 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574009790 con 0x7f258c1014a0 2023-12-14T18:25:00.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.174+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2584009d00 con 0x7f258c1014a0 2023-12-14T18:25:00.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.174+0000 7f2588ff9700 1 -- 172.21.15.93:0/1390089481 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2574010430 con 0x7f258c1014a0 2023-12-14T18:25:00.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.174+0000 7f2588ff9700 1 -- 172.21.15.93:0/1390089481 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2574018680 con 0x7f258c1014a0 2023-12-14T18:25:00.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.174+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f258c10bb80 con 0x7f258c1014a0 2023-12-14T18:25:00.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.175+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f258c1071e0 con 0x7f258c1014a0 2023-12-14T18:25:00.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.176+0000 7f2588ff9700 1 -- 172.21.15.93:0/1390089481 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f25740105a0 con 0x7f258c1014a0 2023-12-14T18:25:00.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.176+0000 7f2588ff9700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25780a11d0 0x7f25780a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:00.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.176+0000 7f258affd700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25780a11d0 0x7f25780a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:00.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.176+0000 7f2588ff9700 1 -- 172.21.15.93:0/1390089481 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2574014070 con 0x7f258c1014a0 2023-12-14T18:25:00.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.177+0000 7f258affd700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25780a11d0 0x7f25780a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f258c067bd0 tx=0x7f257c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:00.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.181+0000 7f2588ff9700 1 -- 172.21.15.93:0/1390089481 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2574051730 con 0x7f258c1014a0 2023-12-14T18:25:00.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:00 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1167192030' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-14T18:25:00.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:00 smithi093 ceph-mon[42070]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:25:00.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:00 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1351863517' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-14T18:25:00.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:00 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1167192030' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-14T18:25:00.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:00 smithi097 ceph-mon[43486]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:25:00.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:00 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1351863517' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-14T18:25:00.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.415+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) v1 -- 0x7f258c02cfe0 con 0x7f258c1014a0 2023-12-14T18:25:00.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:00 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1167192030' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2023-12-14T18:25:00.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:00 smithi160 ceph-mon[43840]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:25:00.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:00 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1351863517' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]: dispatch 2023-12-14T18:25:00.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.752+0000 7f2588ff9700 1 -- 172.21.15.93:0/1390089481 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) v1 ==== 121+0+0 (secure 0 0 0) 0x7f25740961d0 con 0x7f258c1014a0 2023-12-14T18:25:00.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25780a11d0 msgr2=0x7f25780a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25780a11d0 0x7f25780a3680 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f258c067bd0 tx=0x7f257c009260 comp rx=0 tx=0).stop 2023-12-14T18:25:00.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c1014a0 msgr2=0x7f258c06f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c1014a0 0x7f258c06f9a0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f257400e9a0 tx=0x7f257400ecb0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 shutdown_connections 2023-12-14T18:25:00.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f25780a11d0 0x7f25780a3680 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f258c101e30 0x7f258c1a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f258c0fe4d0 0x7f258c06f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 --2- 172.21.15.93:0/1390089481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f258c1014a0 0x7f258c06f9a0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 >> 172.21.15.93:0/1390089481 conn(0x7f258c0f9ed0 msgr2=0x7f258c0fc140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:00.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 shutdown_connections 2023-12-14T18:25:00.757 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.754+0000 7f2591591700 1 -- 172.21.15.93:0/1390089481 wait complete. 2023-12-14T18:25:00.773 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2023-12-14T18:25:00.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.896+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1779947805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 msgr2=0x7fcae8100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.896+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1779947805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 0x7fcae8100770 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fcad8009990 tx=0x7fcad8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.896+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1779947805 shutdown_connections 2023-12-14T18:25:00.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.896+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1779947805 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae8100cb0 0x7fcae8107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.896+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1779947805 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae80ff130 0x7fcae80ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.896+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1779947805 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 0x7fcae8100770 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.896+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1779947805 >> 172.21.15.93:0/1779947805 conn(0x7fcae80fa750 msgr2=0x7fcae80fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:00.900 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.896+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1779947805 shutdown_connections 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.896+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1779947805 wait complete. 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaeea1e700 1 Processor -- start 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaeea1e700 1 -- start start 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaeea1e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae80ff130 0x7fcae8198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaeea1e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 0x7fcae8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaeea1e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae8100cb0 0x7fcae819f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaeea1e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae819fda0 con 0x7fcae8100320 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaeea1e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae819fee0 con 0x7fcae8100cb0 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaeea1e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcae81a0070 con 0x7fcae80ff130 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcae77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 0x7fcae8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcae77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 0x7fcae8198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37982/0 (socket says 172.21.15.93:37982) 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcae77fe700 1 -- 172.21.15.93:0/1856495298 learned_addr learned my addr 172.21.15.93:0/1856495298 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcae7fff700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae80ff130 0x7fcae8198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaecfbb700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae8100cb0 0x7fcae819f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:00.901 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcae77fe700 1 -- 172.21.15.93:0/1856495298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae80ff130 msgr2=0x7fcae8198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcae77fe700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae80ff130 0x7fcae8198220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcae77fe700 1 -- 172.21.15.93:0/1856495298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae8100cb0 msgr2=0x7fcae819f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcae77fe700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae8100cb0 0x7fcae819f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcae77fe700 1 -- 172.21.15.93:0/1856495298 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcad80095f0 con 0x7fcae8100320 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.897+0000 7fcaecfbb700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae8100cb0 0x7fcae819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.898+0000 7fcae77fe700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 0x7fcae8198760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcad8004760 tx=0x7fcad8009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.898+0000 7fcae57fa700 1 -- 172.21.15.93:0/1856495298 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad801d070 con 0x7fcae8100320 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.898+0000 7fcae57fa700 1 -- 172.21.15.93:0/1856495298 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcad8004200 con 0x7fcae8100320 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.898+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcae810ab30 con 0x7fcae8100320 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.898+0000 7fcae57fa700 1 -- 172.21.15.93:0/1856495298 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcad80049f0 con 0x7fcae8100320 2023-12-14T18:25:00.902 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.898+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcae810a9a0 con 0x7fcae8100320 2023-12-14T18:25:00.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.899+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcae81a0770 con 0x7fcae8100320 2023-12-14T18:25:00.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.900+0000 7fcae57fa700 1 -- 172.21.15.93:0/1856495298 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fcad8003d40 con 0x7fcae8100320 2023-12-14T18:25:00.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.900+0000 7fcae57fa700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcad00a11d0 0x7fcad00a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:00.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.900+0000 7fcae57fa700 1 -- 172.21.15.93:0/1856495298 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fcad80cd9c0 con 0x7fcae8100320 2023-12-14T18:25:00.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.900+0000 7fcae7fff700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcad00a11d0 0x7fcad00a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:00.903 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.901+0000 7fcae7fff700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcad00a11d0 0x7fcad00a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006fd0 tx=0x7fcadc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:00.906 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:00.905+0000 7fcae57fa700 1 -- 172.21.15.93:0/1856495298 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcad8096fb0 con 0x7fcae8100320 2023-12-14T18:25:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-12-14T18:25:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-14T18:25:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1351863517' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-14T18:25:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-14T18:25:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:25:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-14T18:25:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-14T18:25:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1390089481' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-14T18:25:01.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] up:active 2023-12-14T18:25:01.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1390089481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-14T18:25:01.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-14T18:25:01.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-14T18:25:01.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: Cluster is now healthy 2023-12-14T18:25:01.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-12-14T18:25:01.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2023-12-14T18:25:01.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:01 smithi093 ceph-mon[42070]: daemon mds.e is now active in filesystem cephfs as rank 2 2023-12-14T18:25:01.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.140+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) v1 -- 0x7fcae81a0d20 con 0x7fcae8100320 2023-12-14T18:25:01.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-12-14T18:25:01.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-14T18:25:01.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1351863517' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1390089481' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] up:active 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1390089481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: Cluster is now healthy 2023-12-14T18:25:01.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-12-14T18:25:01.346 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2023-12-14T18:25:01.346 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-mon[43486]: daemon mds.e is now active in filesystem cephfs as rank 2 2023-12-14T18:25:01.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: pgmap v236: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2023-12-14T18:25:01.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1351863517' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "3"}]': finished 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: daemon mds.i is now active in filesystem cephfs as rank 1 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1390089481' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] up:active 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1390089481' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: daemon mds.e assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2023-12-14T18:25:01.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: Cluster is now healthy 2023-12-14T18:25:01.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 2 up:standby-replay 8 up:standby 2023-12-14T18:25:01.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:creating} 2 up:standby-replay 7 up:standby 2023-12-14T18:25:01.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:01 smithi160 ceph-mon[43840]: daemon mds.e is now active in filesystem cephfs as rank 2 2023-12-14T18:25:01.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.759+0000 7fcae57fa700 1 -- 172.21.15.93:0/1856495298 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v23) v1 ==== 115+0+0 (secure 0 0 0) 0x7fcad809a5d0 con 0x7fcae8100320 2023-12-14T18:25:01.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcad00a11d0 msgr2=0x7fcad00a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:01.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcad00a11d0 0x7fcad00a3680 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcadc006fd0 tx=0x7fcadc008040 comp rx=0 tx=0).stop 2023-12-14T18:25:01.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 msgr2=0x7fcae8198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:01.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 0x7fcae8198760 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fcad8004760 tx=0x7fcad8009f90 comp rx=0 tx=0).stop 2023-12-14T18:25:01.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 shutdown_connections 2023-12-14T18:25:01.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcad00a11d0 0x7fcad00a3680 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcae8100cb0 0x7fcae819f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcae80ff130 0x7fcae8198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 --2- 172.21.15.93:0/1856495298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcae8100320 0x7fcae8198760 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 >> 172.21.15.93:0/1856495298 conn(0x7fcae80fa750 msgr2=0x7fcae8068fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:01.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 shutdown_connections 2023-12-14T18:25:01.765 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.762+0000 7fcaeea1e700 1 -- 172.21.15.93:0/1856495298 wait complete. 2023-12-14T18:25:01.778 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2023-12-14T18:25:01.778 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 '' 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.903+0000 7f5e38a43700 1 -- 172.21.15.93:0/323557038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e34100410 msgr2=0x7f5e34106cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.903+0000 7f5e38a43700 1 --2- 172.21.15.93:0/323557038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e34100410 0x7f5e34106cd0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f5e2c00b0a0 tx=0x7f5e2c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 -- 172.21.15.93:0/323557038 shutdown_connections 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 --2- 172.21.15.93:0/323557038 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e34100410 0x7f5e34106cd0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 --2- 172.21.15.93:0/323557038 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e340fe890 0x7f5e340feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 --2- 172.21.15.93:0/323557038 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e340ffa80 0x7f5e340ffed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 -- 172.21.15.93:0/323557038 >> 172.21.15.93:0/323557038 conn(0x7f5e340f9ed0 msgr2=0x7f5e340fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 -- 172.21.15.93:0/323557038 shutdown_connections 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 -- 172.21.15.93:0/323557038 wait complete. 2023-12-14T18:25:01.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 Processor -- start 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 -- start start 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e340fe890 0x7f5e34194f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e340ffa80 0x7f5e341954d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e34100410 0x7f5e3419c540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.904+0000 7f5e38a43700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3419cb10 con 0x7f5e340fe890 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e38a43700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3419cc50 con 0x7f5e34100410 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e38a43700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e3419cde0 con 0x7f5e340ffa80 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e3259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e340fe890 0x7f5e34194f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e3259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e340fe890 0x7f5e34194f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56790/0 (socket says 172.21.15.93:56790) 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e3259c700 1 -- 172.21.15.93:0/3034026493 learned_addr learned my addr 172.21.15.93:0/3034026493 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:01.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e32d9d700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e34100410 0x7f5e3419c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e31d9b700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e340ffa80 0x7f5e341954d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e3259c700 1 -- 172.21.15.93:0/3034026493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e340ffa80 msgr2=0x7f5e341954d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e3259c700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e340ffa80 0x7f5e341954d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e3259c700 1 -- 172.21.15.93:0/3034026493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e34100410 msgr2=0x7f5e3419c540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e3259c700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e34100410 0x7f5e3419c540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e3259c700 1 -- 172.21.15.93:0/3034026493 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e2c009d00 con 0x7f5e340fe890 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e3259c700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e340fe890 0x7f5e34194f90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800d980 tx=0x7f5e2800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e237fe700 1 -- 172.21.15.93:0/3034026493 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28004020 con 0x7f5e340fe890 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e237fe700 1 -- 172.21.15.93:0/3034026493 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5e28009730 con 0x7f5e340fe890 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e3410aa80 con 0x7f5e340fe890 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e237fe700 1 -- 172.21.15.93:0/3034026493 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e28003a80 con 0x7f5e340fe890 2023-12-14T18:25:01.910 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.905+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e3410a8f0 con 0x7f5e340fe890 2023-12-14T18:25:01.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.906+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e34105d80 con 0x7f5e340fe890 2023-12-14T18:25:01.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.907+0000 7f5e237fe700 1 -- 172.21.15.93:0/3034026493 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5e280098a0 con 0x7f5e340fe890 2023-12-14T18:25:01.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.908+0000 7f5e237fe700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5e1c0a11d0 0x7f5e1c0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:01.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.908+0000 7f5e31d9b700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5e1c0a11d0 0x7f5e1c0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:01.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.908+0000 7f5e237fe700 1 -- 172.21.15.93:0/3034026493 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5e280ccac0 con 0x7f5e340fe890 2023-12-14T18:25:01.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.908+0000 7f5e31d9b700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5e1c0a11d0 0x7f5e1c0a3680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5e24005950 tx=0x7f5e240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:01.913 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:01.911+0000 7f5e237fe700 1 -- 172.21.15.93:0/3034026493 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5e280960a0 con 0x7f5e340fe890 2023-12-14T18:25:02.023 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:25:01 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-b[68560]: 2023-12-14T18:25:01.752+0000 7f94e7277700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2023-12-14T18:25:02.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1856495298' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-14T18:25:02.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] up:active 2023-12-14T18:25:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1856495298' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-14T18:25:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2023-12-14T18:25:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:25:02.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.133+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f5e34061cb0 con 0x7f5e1c0a11d0 2023-12-14T18:25:02.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.174+0000 7f5e237fe700 1 -- 172.21.15.93:0/3034026493 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5e34061cb0 con 0x7f5e1c0a11d0 2023-12-14T18:25:02.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5e1c0a11d0 msgr2=0x7f5e1c0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5e1c0a11d0 0x7f5e1c0a3680 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f5e24005950 tx=0x7f5e240058e0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e340fe890 msgr2=0x7f5e34194f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e340fe890 0x7f5e34194f90 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f5e2800d980 tx=0x7f5e2800dc90 comp rx=0 tx=0).stop 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 shutdown_connections 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5e1c0a11d0 0x7f5e1c0a3680 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5e340fe890 0x7f5e34194f90 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5e34100410 0x7f5e3419c540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 --2- 172.21.15.93:0/3034026493 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5e340ffa80 0x7f5e341954d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 >> 172.21.15.93:0/3034026493 conn(0x7f5e340f9ed0 msgr2=0x7f5e34102ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 shutdown_connections 2023-12-14T18:25:02.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.176+0000 7f5e38a43700 1 -- 172.21.15.93:0/3034026493 wait complete. 2023-12-14T18:25:02.192 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_0 2023-12-14T18:25:02.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.315+0000 7f2ee73d3700 1 -- 172.21.15.93:0/3600906788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 msgr2=0x7f2ee0100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.315+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/3600906788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 0x7f2ee0100780 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2edc009990 tx=0x7f2edc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.320 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.315+0000 7f2ee73d3700 1 -- 172.21.15.93:0/3600906788 shutdown_connections 2023-12-14T18:25:02.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.315+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/3600906788 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ee0100cc0 0x7f2ee0107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.315+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/3600906788 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee00ff140 0x7f2ee00ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.315+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/3600906788 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 0x7f2ee0100780 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.315+0000 7f2ee73d3700 1 -- 172.21.15.93:0/3600906788 >> 172.21.15.93:0/3600906788 conn(0x7f2ee00fa740 msgr2=0x7f2ee00fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:02.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.315+0000 7f2ee73d3700 1 -- 172.21.15.93:0/3600906788 shutdown_connections 2023-12-14T18:25:02.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.315+0000 7f2ee73d3700 1 -- 172.21.15.93:0/3600906788 wait complete. 2023-12-14T18:25:02.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee73d3700 1 Processor -- start 2023-12-14T18:25:02.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee73d3700 1 -- start start 2023-12-14T18:25:02.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee73d3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee00ff140 0x7f2ee0195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee73d3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 0x7f2ee0196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee73d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ee0100cc0 0x7f2ee019d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee73d3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee019dad0 con 0x7f2ee0100330 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee73d3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee019dc10 con 0x7f2ee00ff140 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee73d3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee019dda0 con 0x7f2ee0100cc0 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee496e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 0x7f2ee0196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee496e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 0x7f2ee0196490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56838/0 (socket says 172.21.15.93:56838) 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee496e700 1 -- 172.21.15.93:0/1454747280 learned_addr learned my addr 172.21.15.93:0/1454747280 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee516f700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee00ff140 0x7f2ee0195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee5970700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ee0100cc0 0x7f2ee019d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee496e700 1 -- 172.21.15.93:0/1454747280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ee0100cc0 msgr2=0x7f2ee019d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee496e700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ee0100cc0 0x7f2ee019d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.316+0000 7f2ee496e700 1 -- 172.21.15.93:0/1454747280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee00ff140 msgr2=0x7f2ee0195f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ee496e700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee00ff140 0x7f2ee0195f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ee496e700 1 -- 172.21.15.93:0/1454747280 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2edc0095f0 con 0x7f2ee0100330 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ee5970700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ee0100cc0 0x7f2ee019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ee516f700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee00ff140 0x7f2ee0195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ee496e700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 0x7f2ee0196490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2edc005910 tx=0x7f2edc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ed67fc700 1 -- 172.21.15.93:0/1454747280 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc01d070 con 0x7f2ee0100330 2023-12-14T18:25:02.323 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ed67fc700 1 -- 172.21.15.93:0/1454747280 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2edc0041d0 con 0x7f2ee0100330 2023-12-14T18:25:02.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ed67fc700 1 -- 172.21.15.93:0/1454747280 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2edc00f460 con 0x7f2ee0100330 2023-12-14T18:25:02.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee010ab00 con 0x7f2ee0100330 2023-12-14T18:25:02.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.317+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee010a970 con 0x7f2ee0100330 2023-12-14T18:25:02.324 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.318+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee019e2f0 con 0x7f2ee0100330 2023-12-14T18:25:02.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.319+0000 7f2ed67fc700 1 -- 172.21.15.93:0/1454747280 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2edc0226e0 con 0x7f2ee0100330 2023-12-14T18:25:02.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.320+0000 7f2ed67fc700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2ecc0a1150 0x7f2ecc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.320+0000 7f2ee516f700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2ecc0a1150 0x7f2ecc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.320+0000 7f2ed67fc700 1 -- 172.21.15.93:0/1454747280 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f2edc0ce180 con 0x7f2ee0100330 2023-12-14T18:25:02.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.320+0000 7f2ee516f700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2ecc0a1150 0x7f2ecc0a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0100190 tx=0x7f2ed000a680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:02.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.323+0000 7f2ed67fc700 1 -- 172.21.15.93:0/1454747280 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2edc097770 con 0x7f2ee0100330 2023-12-14T18:25:02.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:02 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1856495298' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-14T18:25:02.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:02 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] up:active 2023-12-14T18:25:02.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:02 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1856495298' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-14T18:25:02.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:02 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2023-12-14T18:25:02.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:02 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:25:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:02 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1856495298' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2023-12-14T18:25:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:02 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] up:active 2023-12-14T18:25:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:02 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1856495298' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2023-12-14T18:25:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:02 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 2 up:standby-replay 7 up:standby 2023-12-14T18:25:02.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:02 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:25:02.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.544+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) v1 -- 0x7f2ee00694d0 con 0x7f2ecc0a1150 2023-12-14T18:25:02.550 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.548+0000 7f2ed67fc700 1 -- 172.21.15.93:0/1454747280 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7f2ee00694d0 con 0x7f2ecc0a1150 2023-12-14T18:25:02.550 INFO:teuthology.orchestra.run.smithi093.stdout:/volumes/_nogroup/sv_0/3514a361-3b53-4473-b796-e3aeef97c222 2023-12-14T18:25:02.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2ecc0a1150 msgr2=0x7f2ecc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2ecc0a1150 0x7f2ecc0a3600 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f2ee0100190 tx=0x7f2ed000a680 comp rx=0 tx=0).stop 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 msgr2=0x7f2ee0196490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 0x7f2ee0196490 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f2edc005910 tx=0x7f2edc009f90 comp rx=0 tx=0).stop 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 shutdown_connections 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2ecc0a1150 0x7f2ecc0a3600 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2ee0100cc0 0x7f2ee019d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2ee00ff140 0x7f2ee0195f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 --2- 172.21.15.93:0/1454747280 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2ee0100330 0x7f2ee0196490 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 >> 172.21.15.93:0/1454747280 conn(0x7f2ee00fa740 msgr2=0x7f2ee0068dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 shutdown_connections 2023-12-14T18:25:02.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.550+0000 7f2ee73d3700 1 -- 172.21.15.93:0/1454747280 wait complete. 2023-12-14T18:25:02.563 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:25:02.138+0000 7fda4b29f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:25:02.563 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:25:02.138+0000 7fda4b29f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:25:02.563 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:25:02.138+0000 7fda4b29f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:25:02.563 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:25:02.138+0000 7fda4b29f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:25:02.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:25:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:25:02.138+0000 7fda4b29f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:25:02.565 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 '' 2023-12-14T18:25:02.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.689+0000 7f91e25c9700 1 -- 172.21.15.93:0/290403242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc100310 msgr2=0x7f91dc100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.689+0000 7f91e25c9700 1 --2- 172.21.15.93:0/290403242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc100310 0x7f91dc100760 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009990 tx=0x7f91d0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.689+0000 7f91e25c9700 1 -- 172.21.15.93:0/290403242 shutdown_connections 2023-12-14T18:25:02.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.689+0000 7f91e25c9700 1 --2- 172.21.15.93:0/290403242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc100ca0 0x7f91dc107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.689+0000 7f91e25c9700 1 --2- 172.21.15.93:0/290403242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc0ff120 0x7f91dc0ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.689+0000 7f91e25c9700 1 --2- 172.21.15.93:0/290403242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc100310 0x7f91dc100760 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.689+0000 7f91e25c9700 1 -- 172.21.15.93:0/290403242 >> 172.21.15.93:0/290403242 conn(0x7f91dc0fa740 msgr2=0x7f91dc0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:02.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.689+0000 7f91e25c9700 1 -- 172.21.15.93:0/290403242 shutdown_connections 2023-12-14T18:25:02.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.689+0000 7f91e25c9700 1 -- 172.21.15.93:0/290403242 wait complete. 2023-12-14T18:25:02.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91e25c9700 1 Processor -- start 2023-12-14T18:25:02.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91e25c9700 1 -- start start 2023-12-14T18:25:02.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91e25c9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc0ff120 0x7f91dc069b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91e25c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc100310 0x7f91dc19ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91e25c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc100ca0 0x7f91dc19d270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91e25c9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc19da40 con 0x7f91dc0ff120 2023-12-14T18:25:02.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91e25c9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc19dbb0 con 0x7f91dc100310 2023-12-14T18:25:02.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91e25c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91dc19dd70 con 0x7f91dc100ca0 2023-12-14T18:25:02.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc0ff120 0x7f91dc069b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91dbfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc0ff120 0x7f91dc069b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56850/0 (socket says 172.21.15.93:56850) 2023-12-14T18:25:02.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91dbfff700 1 -- 172.21.15.93:0/1512387425 learned_addr learned my addr 172.21.15.93:0/1512387425 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:02.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91e0b66700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc100ca0 0x7f91dc19d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91db7fe700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc100310 0x7f91dc19ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91dbfff700 1 -- 172.21.15.93:0/1512387425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc100ca0 msgr2=0x7f91dc19d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91dbfff700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc100ca0 0x7f91dc19d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91dbfff700 1 -- 172.21.15.93:0/1512387425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc100310 msgr2=0x7f91dc19ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91dbfff700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc100310 0x7f91dc19ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91dbfff700 1 -- 172.21.15.93:0/1512387425 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91d00095f0 con 0x7f91dc0ff120 2023-12-14T18:25:02.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.690+0000 7f91dbfff700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc0ff120 0x7f91dc069b10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c990 tx=0x7f91cc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.691+0000 7f91d97fa700 1 -- 172.21.15.93:0/1512387425 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc007b00 con 0x7f91dc0ff120 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.691+0000 7f91d97fa700 1 -- 172.21.15.93:0/1512387425 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f91cc007c60 con 0x7f91dc0ff120 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.691+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91dc10aa80 con 0x7f91dc0ff120 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.691+0000 7f91d97fa700 1 -- 172.21.15.93:0/1512387425 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91cc018660 con 0x7f91dc0ff120 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.691+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91dc10a8f0 con 0x7f91dc0ff120 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.692+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91dc19e400 con 0x7f91dc0ff120 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.693+0000 7f91d97fa700 1 -- 172.21.15.93:0/1512387425 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f91cc00f450 con 0x7f91dc0ff120 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.693+0000 7f91d97fa700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f91c40a1150 0x7f91c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.693+0000 7f91db7fe700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f91c40a1150 0x7f91c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.693+0000 7f91d97fa700 1 -- 172.21.15.93:0/1512387425 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f91cc0ccb10 con 0x7f91dc0ff120 2023-12-14T18:25:02.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.694+0000 7f91db7fe700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f91c40a1150 0x7f91c40a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91d000b5c0 tx=0x7f91d0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:02.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.697+0000 7f91d97fa700 1 -- 172.21.15.93:0/1512387425 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f91cc096120 con 0x7f91dc0ff120 2023-12-14T18:25:02.919 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.917+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}) v1 -- 0x7f91dc19e9b0 con 0x7f91c40a1150 2023-12-14T18:25:02.945 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.944+0000 7f91d97fa700 1 -- 172.21.15.93:0/1512387425 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f91dc19e9b0 con 0x7f91c40a1150 2023-12-14T18:25:02.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f91c40a1150 msgr2=0x7f91c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f91c40a1150 0x7f91c40a3600 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f91d000b5c0 tx=0x7f91d0005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc0ff120 msgr2=0x7f91dc069b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:02.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc0ff120 0x7f91dc069b10 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f91cc00c990 tx=0x7f91cc00cd50 comp rx=0 tx=0).stop 2023-12-14T18:25:02.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 shutdown_connections 2023-12-14T18:25:02.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f91c40a1150 0x7f91c40a3600 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f91dc0ff120 0x7f91dc069b10 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f91dc100ca0 0x7f91dc19d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 --2- 172.21.15.93:0/1512387425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f91dc100310 0x7f91dc19ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:02.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 >> 172.21.15.93:0/1512387425 conn(0x7f91dc0fa740 msgr2=0x7f91dc0fc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:02.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 shutdown_connections 2023-12-14T18:25:02.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:02.946+0000 7f91e25c9700 1 -- 172.21.15.93:0/1512387425 wait complete. 2023-12-14T18:25:02.960 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs sv_1 2023-12-14T18:25:03.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.084+0000 7fd3119ae700 1 -- 172.21.15.93:0/1592320077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 msgr2=0x7fd30c108670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.084+0000 7fd3119ae700 1 --2- 172.21.15.93:0/1592320077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 0x7fd30c108670 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fd30400b0a0 tx=0x7fd30400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.084+0000 7fd3119ae700 1 -- 172.21.15.93:0/1592320077 shutdown_connections 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.084+0000 7fd3119ae700 1 --2- 172.21.15.93:0/1592320077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 0x7fd30c108670 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.084+0000 7fd3119ae700 1 --2- 172.21.15.93:0/1592320077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c100230 0x7fd30c100640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.084+0000 7fd3119ae700 1 --2- 172.21.15.93:0/1592320077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd30c101420 0x7fd30c101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.084+0000 7fd3119ae700 1 -- 172.21.15.93:0/1592320077 >> 172.21.15.93:0/1592320077 conn(0x7fd30c0fb850 msgr2=0x7fd30c0fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.084+0000 7fd3119ae700 1 -- 172.21.15.93:0/1592320077 shutdown_connections 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.084+0000 7fd3119ae700 1 -- 172.21.15.93:0/1592320077 wait complete. 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd3119ae700 1 Processor -- start 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd3119ae700 1 -- start start 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd3119ae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd30c100230 0x7fd30c0748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd3119ae700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c101420 0x7fd30c072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd3119ae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 0x7fd30c073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd3119ae700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c073bc0 con 0x7fd30c101db0 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd3119ae700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c073d30 con 0x7fd30c101420 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd3119ae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd30c1a0080 con 0x7fd30c100230 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 0x7fd30c073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd30b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 0x7fd30c073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56868/0 (socket says 172.21.15.93:56868) 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.085+0000 7fd30b7fe700 1 -- 172.21.15.93:0/3800728050 learned_addr learned my addr 172.21.15.93:0/3800728050 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30affd700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd30c100230 0x7fd30c0748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30a7fc700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c101420 0x7fd30c072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30b7fe700 1 -- 172.21.15.93:0/3800728050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd30c100230 msgr2=0x7fd30c0748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30b7fe700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd30c100230 0x7fd30c0748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30b7fe700 1 -- 172.21.15.93:0/3800728050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c101420 msgr2=0x7fd30c072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30b7fe700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c101420 0x7fd30c072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30b7fe700 1 -- 172.21.15.93:0/3800728050 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd304009d00 con 0x7fd30c101db0 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30affd700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd30c100230 0x7fd30c0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:03.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30a7fc700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c101420 0x7fd30c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:03.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd30b7fe700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 0x7fd30c073420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd30400d010 tx=0x7fd304004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:03.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd3109ac700 1 -- 172.21.15.93:0/3800728050 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd30400e040 con 0x7fd30c101db0 2023-12-14T18:25:03.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd3109ac700 1 -- 172.21.15.93:0/3800728050 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd304009440 con 0x7fd30c101db0 2023-12-14T18:25:03.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd30c10bc00 con 0x7fd30c101db0 2023-12-14T18:25:03.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd3109ac700 1 -- 172.21.15.93:0/3800728050 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd304004280 con 0x7fd30c101db0 2023-12-14T18:25:03.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.086+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd30c10ba70 con 0x7fd30c101db0 2023-12-14T18:25:03.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.087+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd30c107070 con 0x7fd30c101db0 2023-12-14T18:25:03.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.088+0000 7fd3109ac700 1 -- 172.21.15.93:0/3800728050 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fd30401d720 con 0x7fd30c101db0 2023-12-14T18:25:03.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.089+0000 7fd3109ac700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd2f40a1150 0x7fd2f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.089+0000 7fd30affd700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd2f40a1150 0x7fd2f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.089+0000 7fd3109ac700 1 -- 172.21.15.93:0/3800728050 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd3040ce180 con 0x7fd30c101db0 2023-12-14T18:25:03.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.089+0000 7fd30affd700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd2f40a1150 0x7fd2f40a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc005fd0 tx=0x7fd2fc005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:03.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.093+0000 7fd3109ac700 1 -- 172.21.15.93:0/3800728050 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd304097710 con 0x7fd30c101db0 2023-12-14T18:25:03.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:03 smithi093 ceph-mon[42070]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-12-14T18:25:03.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:03 smithi093 ceph-mon[42070]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:03.315 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.314+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fd30c061b20 con 0x7fd2f40a1150 2023-12-14T18:25:03.319 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.317+0000 7fd3109ac700 1 -- 172.21.15.93:0/3800728050 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+60 (secure 0 0 0) 0x7fd30c061b20 con 0x7fd2f40a1150 2023-12-14T18:25:03.319 INFO:teuthology.orchestra.run.smithi093.stdout:/volumes/_nogroup/sv_1/17dc08c9-cb20-4697-9d38-a6077c301cb3 2023-12-14T18:25:03.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd2f40a1150 msgr2=0x7fd2f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd2f40a1150 0x7fd2f40a3600 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fd2fc005fd0 tx=0x7fd2fc005e40 comp rx=0 tx=0).stop 2023-12-14T18:25:03.321 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 msgr2=0x7fd30c073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 0x7fd30c073420 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fd30400d010 tx=0x7fd304004d30 comp rx=0 tx=0).stop 2023-12-14T18:25:03.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 shutdown_connections 2023-12-14T18:25:03.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd2f40a1150 0x7fd2f40a3600 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd30c101db0 0x7fd30c073420 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd30c100230 0x7fd30c0748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 --2- 172.21.15.93:0/3800728050 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd30c101420 0x7fd30c072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 >> 172.21.15.93:0/3800728050 conn(0x7fd30c0fb850 msgr2=0x7fd30c0fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:03.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 shutdown_connections 2023-12-14T18:25:03.322 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.319+0000 7fd3119ae700 1 -- 172.21.15.93:0/3800728050 wait complete. 2023-12-14T18:25:03.333 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:03.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:03 smithi097 ceph-mon[43486]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-12-14T18:25:03.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:03 smithi097 ceph-mon[43486]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:03.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.453+0000 7fe081872700 1 -- 172.21.15.93:0/2019358466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 msgr2=0x7fe07c107590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.453+0000 7fe081872700 1 --2- 172.21.15.93:0/2019358466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 0x7fe07c107590 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe07400b0a0 tx=0x7fe07400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.453+0000 7fe081872700 1 -- 172.21.15.93:0/2019358466 shutdown_connections 2023-12-14T18:25:03.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.453+0000 7fe081872700 1 --2- 172.21.15.93:0/2019358466 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 0x7fe07c107590 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.453+0000 7fe081872700 1 --2- 172.21.15.93:0/2019358466 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe07c0ff150 0x7fe07c0ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.453+0000 7fe081872700 1 --2- 172.21.15.93:0/2019358466 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe07c100340 0x7fe07c100790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.453+0000 7fe081872700 1 -- 172.21.15.93:0/2019358466 >> 172.21.15.93:0/2019358466 conn(0x7fe07c0fa750 msgr2=0x7fe07c0fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:03.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.453+0000 7fe081872700 1 -- 172.21.15.93:0/2019358466 shutdown_connections 2023-12-14T18:25:03.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.454+0000 7fe081872700 1 -- 172.21.15.93:0/2019358466 wait complete. 2023-12-14T18:25:03.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.454+0000 7fe081872700 1 Processor -- start 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.454+0000 7fe081872700 1 -- start start 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe081872700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe07c0ff150 0x7fe07c069be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe081872700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe07c100340 0x7fe07c19ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe081872700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 0x7fe07c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe081872700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19dc00 con 0x7fe07c100cd0 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe081872700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19dd70 con 0x7fe07c0ff150 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe081872700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe07c19df30 con 0x7fe07c100340 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 0x7fe07c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 0x7fe07c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56876/0 (socket says 172.21.15.93:56876) 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07b7fe700 1 -- 172.21.15.93:0/1612166527 learned_addr learned my addr 172.21.15.93:0/1612166527 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07affd700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe07c0ff150 0x7fe07c069be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07a7fc700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe07c100340 0x7fe07c19ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07b7fe700 1 -- 172.21.15.93:0/1612166527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe07c100340 msgr2=0x7fe07c19ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07b7fe700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe07c100340 0x7fe07c19ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07b7fe700 1 -- 172.21.15.93:0/1612166527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe07c0ff150 msgr2=0x7fe07c069be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07b7fe700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe07c0ff150 0x7fe07c069be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07b7fe700 1 -- 172.21.15.93:0/1612166527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe074009d00 con 0x7fe07c100cd0 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07affd700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe07c0ff150 0x7fe07c069be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07a7fc700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe07c100340 0x7fe07c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe07b7fe700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 0x7fe07c19d460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe074007950 tx=0x7fe074007cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe080870700 1 -- 172.21.15.93:0/1612166527 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07400e040 con 0x7fe07c100cd0 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe080870700 1 -- 172.21.15.93:0/1612166527 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe07401d480 con 0x7fe07c100cd0 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.455+0000 7fe080870700 1 -- 172.21.15.93:0/1612166527 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07401b6b0 con 0x7fe07c100cd0 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.456+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe07c10aac0 con 0x7fe07c100cd0 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.456+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe07c10a930 con 0x7fe07c100cd0 2023-12-14T18:25:03.460 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.457+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe07c1066a0 con 0x7fe07c100cd0 2023-12-14T18:25:03.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.457+0000 7fe080870700 1 -- 172.21.15.93:0/1612166527 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe07401d5f0 con 0x7fe07c100cd0 2023-12-14T18:25:03.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.458+0000 7fe080870700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe0640a1150 0x7fe0640a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.458+0000 7fe080870700 1 -- 172.21.15.93:0/1612166527 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fe0740ce100 con 0x7fe07c100cd0 2023-12-14T18:25:03.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.458+0000 7fe07affd700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe0640a1150 0x7fe0640a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.458+0000 7fe07affd700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe0640a1150 0x7fe0640a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe07c1001a0 tx=0x7fe06c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:03.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.463+0000 7fe080870700 1 -- 172.21.15.93:0/1612166527 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe0740976f0 con 0x7fe07c100cd0 2023-12-14T18:25:03.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:03 smithi160 ceph-mon[43840]: pgmap v237: 129 pgs: 129 active+clean; 581 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 597 B/s rd, 1.2 KiB/s wr, 3 op/s 2023-12-14T18:25:03.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:03 smithi160 ceph-mon[43840]: from='client.14871 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:03.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.695+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe07c061820 con 0x7fe07c100cd0 2023-12-14T18:25:03.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.696+0000 7fe080870700 1 -- 172.21.15.93:0/1612166527 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10799 (secure 0 0 0) 0x7fe07409ad10 con 0x7fe07c100cd0 2023-12-14T18:25:03.698 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:03.698 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:25:01.760987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:25:03.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe0640a1150 msgr2=0x7fe0640a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe0640a1150 0x7fe0640a3600 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fe07c1001a0 tx=0x7fe06c009260 comp rx=0 tx=0).stop 2023-12-14T18:25:03.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 msgr2=0x7fe07c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 0x7fe07c19d460 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe074007950 tx=0x7fe074007cf0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 shutdown_connections 2023-12-14T18:25:03.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe0640a1150 0x7fe0640a3600 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe07c100cd0 0x7fe07c19d460 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe07c0ff150 0x7fe07c069be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 --2- 172.21.15.93:0/1612166527 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe07c100340 0x7fe07c19ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 >> 172.21.15.93:0/1612166527 conn(0x7fe07c0fa750 msgr2=0x7fe07c1037e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:03.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 shutdown_connections 2023-12-14T18:25:03.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.698+0000 7fe081872700 1 -- 172.21.15.93:0/1612166527 wait complete. 2023-12-14T18:25:03.701 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 24 2023-12-14T18:25:03.712 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:25:03.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.832+0000 7fb98beda700 1 -- 172.21.15.93:0/3197418237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb984100c60 msgr2=0x7fb984107520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.832+0000 7fb98beda700 1 --2- 172.21.15.93:0/3197418237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb984100c60 0x7fb984107520 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fb97c00b0a0 tx=0x7fb97c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.832+0000 7fb98beda700 1 -- 172.21.15.93:0/3197418237 shutdown_connections 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.832+0000 7fb98beda700 1 --2- 172.21.15.93:0/3197418237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb984100c60 0x7fb984107520 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.832+0000 7fb98beda700 1 --2- 172.21.15.93:0/3197418237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb9840ff0e0 0x7fb9840ff4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.832+0000 7fb98beda700 1 --2- 172.21.15.93:0/3197418237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9841002d0 0x7fb984100720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.832+0000 7fb98beda700 1 -- 172.21.15.93:0/3197418237 >> 172.21.15.93:0/3197418237 conn(0x7fb9840fa700 msgr2=0x7fb9840fcb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.832+0000 7fb98beda700 1 -- 172.21.15.93:0/3197418237 shutdown_connections 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.832+0000 7fb98beda700 1 -- 172.21.15.93:0/3197418237 wait complete. 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98beda700 1 Processor -- start 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98beda700 1 -- start start 2023-12-14T18:25:03.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98beda700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9840ff0e0 0x7fb984196030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98beda700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9841002d0 0x7fb984196570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98beda700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb984100c60 0x7fb98419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98beda700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98419dbb0 con 0x7fb9841002d0 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98beda700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98419dcf0 con 0x7fb984100c60 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98beda700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb98419de80 con 0x7fb9840ff0e0 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989475700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9841002d0 0x7fb984196570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989475700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9841002d0 0x7fb984196570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56892/0 (socket says 172.21.15.93:56892) 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989475700 1 -- 172.21.15.93:0/4265482339 learned_addr learned my addr 172.21.15.93:0/4265482339 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98a477700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb984100c60 0x7fb98419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989c76700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9840ff0e0 0x7fb984196030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989475700 1 -- 172.21.15.93:0/4265482339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9840ff0e0 msgr2=0x7fb984196030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989475700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9840ff0e0 0x7fb984196030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989475700 1 -- 172.21.15.93:0/4265482339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb984100c60 msgr2=0x7fb98419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989475700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb984100c60 0x7fb98419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989475700 1 -- 172.21.15.93:0/4265482339 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb97c009d00 con 0x7fb9841002d0 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98a477700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb984100c60 0x7fb98419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb989475700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9841002d0 0x7fb984196570 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb98000d980 tx=0x7fb98000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb97affd700 1 -- 172.21.15.93:0/4265482339 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb98000bb60 con 0x7fb9841002d0 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.833+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb98410aa60 con 0x7fb9841002d0 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.834+0000 7fb97affd700 1 -- 172.21.15.93:0/4265482339 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb980007640 con 0x7fb9841002d0 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.834+0000 7fb97affd700 1 -- 172.21.15.93:0/4265482339 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb980004e90 con 0x7fb9841002d0 2023-12-14T18:25:03.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.834+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb98410a8d0 con 0x7fb9841002d0 2023-12-14T18:25:03.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.835+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb98419e5c0 con 0x7fb9841002d0 2023-12-14T18:25:03.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.836+0000 7fb97affd700 1 -- 172.21.15.93:0/4265482339 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 16) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb98000bcc0 con 0x7fb9841002d0 2023-12-14T18:25:03.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.836+0000 7fb97affd700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb9700a1150 0x7fb9700a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:03.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.836+0000 7fb989c76700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb9700a1150 0x7fb9700a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:03.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.836+0000 7fb97affd700 1 -- 172.21.15.93:0/4265482339 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb9800cced0 con 0x7fb9841002d0 2023-12-14T18:25:03.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.837+0000 7fb989c76700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb9700a1150 0x7fb9700a3600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb974005950 tx=0x7fb9740058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:03.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:03.839+0000 7fb97affd700 1 -- 172.21.15.93:0/4265482339 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb980099ec0 con 0x7fb9841002d0 2023-12-14T18:25:04.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.030+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb98419eb70 con 0x7fb9841002d0 2023-12-14T18:25:04.038 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.037+0000 7fb97affd700 1 -- 172.21.15.93:0/4265482339 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb980096800 con 0x7fb9841002d0 2023-12-14T18:25:04.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.039+0000 7fb97affd700 1 -- 172.21.15.93:0/4265482339 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7fb98008ea60 con 0x7fb9841002d0 2023-12-14T18:25:04.042 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:04.042 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:25:04.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb9700a1150 msgr2=0x7fb9700a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb9700a1150 0x7fb9700a3600 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fb974005950 tx=0x7fb9740058e0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9841002d0 msgr2=0x7fb984196570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9841002d0 0x7fb984196570 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fb98000d980 tx=0x7fb98000dd40 comp rx=0 tx=0).stop 2023-12-14T18:25:04.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 shutdown_connections 2023-12-14T18:25:04.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb9700a1150 0x7fb9700a3600 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb984100c60 0x7fb98419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb9840ff0e0 0x7fb984196030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 --2- 172.21.15.93:0/4265482339 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb9841002d0 0x7fb984196570 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 >> 172.21.15.93:0/4265482339 conn(0x7fb9840fa700 msgr2=0x7fb984067e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:04.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 shutdown_connections 2023-12-14T18:25:04.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.042+0000 7fb98beda700 1 -- 172.21.15.93:0/4265482339 wait complete. 2023-12-14T18:25:04.056 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-14T18:25:04.057 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-14T18:25:04.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.176+0000 7f0920d8e700 1 -- 172.21.15.93:0/278391652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c0ff110 msgr2=0x7f091c0ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.176+0000 7f0920d8e700 1 --2- 172.21.15.93:0/278391652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c0ff110 0x7f091c0ff520 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f0910009990 tx=0x7f0910009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.176+0000 7f0920d8e700 1 -- 172.21.15.93:0/278391652 shutdown_connections 2023-12-14T18:25:04.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.176+0000 7f0920d8e700 1 --2- 172.21.15.93:0/278391652 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c0ff110 0x7f091c0ff520 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.176+0000 7f0920d8e700 1 --2- 172.21.15.93:0/278391652 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f091c100c90 0x7f091c107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.176+0000 7f0920d8e700 1 --2- 172.21.15.93:0/278391652 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091c100300 0x7f091c100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.176+0000 7f0920d8e700 1 -- 172.21.15.93:0/278391652 >> 172.21.15.93:0/278391652 conn(0x7f091c0fa6f0 msgr2=0x7f091c0fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:04.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.176+0000 7f0920d8e700 1 -- 172.21.15.93:0/278391652 shutdown_connections 2023-12-14T18:25:04.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.176+0000 7f0920d8e700 1 -- 172.21.15.93:0/278391652 wait complete. 2023-12-14T18:25:04.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.177+0000 7f0920d8e700 1 Processor -- start 2023-12-14T18:25:04.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.177+0000 7f0920d8e700 1 -- start start 2023-12-14T18:25:04.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0920d8e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091c0ff110 0x7f091c194f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:04.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0920d8e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c100300 0x7f091c195470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:04.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0920d8e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f091c100c90 0x7f091c19c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:04.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0920d8e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c19cab0 con 0x7f091c100300 2023-12-14T18:25:04.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0920d8e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c19cbf0 con 0x7f091c0ff110 2023-12-14T18:25:04.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0920d8e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f091c19cd80 con 0x7f091c100c90 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0919d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c100300 0x7f091c195470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0919d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c100300 0x7f091c195470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56914/0 (socket says 172.21.15.93:56914) 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0919d9b700 1 -- 172.21.15.93:0/2373870663 learned_addr learned my addr 172.21.15.93:0/2373870663 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f091a59c700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091c0ff110 0x7f091c194f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f091ad9d700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f091c100c90 0x7f091c19c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0919d9b700 1 -- 172.21.15.93:0/2373870663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f091c100c90 msgr2=0x7f091c19c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0919d9b700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f091c100c90 0x7f091c19c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0919d9b700 1 -- 172.21.15.93:0/2373870663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091c0ff110 msgr2=0x7f091c194f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0919d9b700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091c0ff110 0x7f091c194f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0919d9b700 1 -- 172.21.15.93:0/2373870663 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09100095f0 con 0x7f091c100300 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f091ad9d700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f091c100c90 0x7f091c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0919d9b700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c100300 0x7f091c195470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f090c00d950 tx=0x7f090c00dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f090b7fe700 1 -- 172.21.15.93:0/2373870663 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c009940 con 0x7f091c100300 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f090b7fe700 1 -- 172.21.15.93:0/2373870663 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f090c010460 con 0x7f091c100300 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f091c10aad0 con 0x7f091c100300 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f090b7fe700 1 -- 172.21.15.93:0/2373870663 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f090c00f620 con 0x7f091c100300 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.178+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f091c10a940 con 0x7f091c100300 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.179+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f091c0679f0 con 0x7f091c100300 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.180+0000 7f090b7fe700 1 -- 172.21.15.93:0/2373870663 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f090c0108d0 con 0x7f091c100300 2023-12-14T18:25:04.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.181+0000 7f090b7fe700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f09040a11e0 0x7f09040a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:04.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.181+0000 7f091a59c700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f09040a11e0 0x7f09040a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:04.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.181+0000 7f090b7fe700 1 -- 172.21.15.93:0/2373870663 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f090c0ccb60 con 0x7f091c100300 2023-12-14T18:25:04.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.181+0000 7f091a59c700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f09040a11e0 0x7f09040a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f091000b5c0 tx=0x7f0910005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:04.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.186+0000 7f090b7fe700 1 -- 172.21.15.93:0/2373870663 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f090c095f00 con 0x7f091c100300 2023-12-14T18:25:04.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:04 smithi093 ceph-mon[42070]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:04 smithi093 ceph-mon[42070]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:04 smithi093 ceph-mon[42070]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:04 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1612166527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:04.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:04 smithi097 ceph-mon[43486]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:04 smithi097 ceph-mon[43486]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:04 smithi097 ceph-mon[43486]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:04 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1612166527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:04.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.373+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f091c060f30 con 0x7f09040a11e0 2023-12-14T18:25:04.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.385+0000 7f090b7fe700 1 -- 172.21.15.93:0/2373870663 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418218 (secure 0 0 0) 0x7f091c060f30 con 0x7f09040a11e0 2023-12-14T18:25:04.394 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f09040a11e0 msgr2=0x7f09040a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f09040a11e0 0x7f09040a3690 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f091000b5c0 tx=0x7f0910005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c100300 msgr2=0x7f091c195470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c100300 0x7f091c195470 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f090c00d950 tx=0x7f090c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:25:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 shutdown_connections 2023-12-14T18:25:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f09040a11e0 0x7f09040a3690 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f091c100c90 0x7f091c19c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f091c0ff110 0x7f091c194f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 --2- 172.21.15.93:0/2373870663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f091c100300 0x7f091c195470 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 >> 172.21.15.93:0/2373870663 conn(0x7f091c0fa6f0 msgr2=0x7f091c0fca10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 shutdown_connections 2023-12-14T18:25:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.389+0000 7f0920d8e700 1 -- 172.21.15.93:0/2373870663 wait complete. 2023-12-14T18:25:04.402 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-12-14T18:25:04.403 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2023-12-14T18:25:03.951486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597112,"num_objects":55,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":150,"num_read_kb":133,"num_write":299,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":78888,"kb_used_data":11016,"kb_used_omap":0,"kb_used_meta":67776,"kb_avail":1124862936,"statfs":{"total":1151940427776,"available":1151859646464,"internally_reserved":0,"allocated":11280384,"data_stored":7279949,"data_compressed":246693,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":0,"internal_metadata":69402624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2294,"num_objects":31,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":71,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003432"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939919+0000","last_change":"2023-12-14T18:24:36.587133+0000","last_active":"2023-12-14T18:24:41.939919+0000","last_peered":"2023-12-14T18:24:41.939919+0000","last_clean":"2023-12-14T18:24:41.939919+0000","last_became_active":"2023-12-14T18:24:36.586222+0000","last_became_peered":"2023-12-14T18:24:36.586222+0000","last_unstale":"2023-12-14T18:24:41.939919+0000","last_undegraded":"2023-12-14T18:24:41.939919+0000","last_fullsized":"2023-12-14T18:24:41.939919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:51:38.922525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939529+0000","last_change":"2023-12-14T18:24:36.589027+0000","last_active":"2023-12-14T18:24:41.939529+0000","last_peered":"2023-12-14T18:24:41.939529+0000","last_clean":"2023-12-14T18:24:41.939529+0000","last_became_active":"2023-12-14T18:24:36.588785+0000","last_became_peered":"2023-12-14T18:24:36.588785+0000","last_unstale":"2023-12-14T18:24:41.939529+0000","last_undegraded":"2023-12-14T18:24:41.939529+0000","last_fullsized":"2023-12-14T18:24:41.939529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:42.987957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939327+0000","last_change":"2023-12-14T18:24:36.584015+0000","last_active":"2023-12-14T18:24:41.939327+0000","last_peered":"2023-12-14T18:24:41.939327+0000","last_clean":"2023-12-14T18:24:41.939327+0000","last_became_active":"2023-12-14T18:24:36.583735+0000","last_became_peered":"2023-12-14T18:24:36.583735+0000","last_unstale":"2023-12-14T18:24:41.939327+0000","last_undegraded":"2023-12-14T18:24:41.939327+0000","last_fullsized":"2023-12-14T18:24:41.939327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:20:02.823540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940899+0000","last_change":"2023-12-14T18:24:36.584553+0000","last_active":"2023-12-14T18:24:41.940899+0000","last_peered":"2023-12-14T18:24:41.940899+0000","last_clean":"2023-12-14T18:24:41.940899+0000","last_became_active":"2023-12-14T18:24:36.583827+0000","last_became_peered":"2023-12-14T18:24:36.583827+0000","last_unstale":"2023-12-14T18:24:41.940899+0000","last_undegraded":"2023-12-14T18:24:41.940899+0000","last_fullsized":"2023-12-14T18:24:41.940899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:32.928164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095366+0000","last_change":"2023-12-14T18:24:36.578128+0000","last_active":"2023-12-14T18:24:41.095366+0000","last_peered":"2023-12-14T18:24:41.095366+0000","last_clean":"2023-12-14T18:24:41.095366+0000","last_became_active":"2023-12-14T18:24:36.577784+0000","last_became_peered":"2023-12-14T18:24:36.577784+0000","last_unstale":"2023-12-14T18:24:41.095366+0000","last_undegraded":"2023-12-14T18:24:41.095366+0000","last_fullsized":"2023-12-14T18:24:41.095366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:17.890524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940347+0000","last_change":"2023-12-14T18:24:36.588666+0000","last_active":"2023-12-14T18:24:41.940347+0000","last_peered":"2023-12-14T18:24:41.940347+0000","last_clean":"2023-12-14T18:24:41.940347+0000","last_became_active":"2023-12-14T18:24:36.588291+0000","last_became_peered":"2023-12-14T18:24:36.588291+0000","last_unstale":"2023-12-14T18:24:41.940347+0000","last_undegraded":"2023-12-14T18:24:41.940347+0000","last_fullsized":"2023-12-14T18:24:41.940347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:10:59.591257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941626+0000","last_change":"2023-12-14T18:24:36.586637+0000","last_active":"2023-12-14T18:24:41.941626+0000","last_peered":"2023-12-14T18:24:41.941626+0000","last_clean":"2023-12-14T18:24:41.941626+0000","last_became_active":"2023-12-14T18:24:36.583055+0000","last_became_peered":"2023-12-14T18:24:36.583055+0000","last_unstale":"2023-12-14T18:24:41.941626+0000","last_undegraded":"2023-12-14T18:24:41.941626+0000","last_fullsized":"2023-12-14T18:24:41.941626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:31.010603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998916+0000","last_change":"2023-12-14T18:24:36.580206+0000","last_active":"2023-12-14T18:24:40.998916+0000","last_peered":"2023-12-14T18:24:40.998916+0000","last_clean":"2023-12-14T18:24:40.998916+0000","last_became_active":"2023-12-14T18:24:36.579904+0000","last_became_peered":"2023-12-14T18:24:36.579904+0000","last_unstale":"2023-12-14T18:24:40.998916+0000","last_undegraded":"2023-12-14T18:24:40.998916+0000","last_fullsized":"2023-12-14T18:24:40.998916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:15.182919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000918+0000","last_change":"2023-12-14T18:24:36.589890+0000","last_active":"2023-12-14T18:24:41.000918+0000","last_peered":"2023-12-14T18:24:41.000918+0000","last_clean":"2023-12-14T18:24:41.000918+0000","last_became_active":"2023-12-14T18:24:36.588999+0000","last_became_peered":"2023-12-14T18:24:36.588999+0000","last_unstale":"2023-12-14T18:24:41.000918+0000","last_undegraded":"2023-12-14T18:24:41.000918+0000","last_fullsized":"2023-12-14T18:24:41.000918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:59:07.801717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941211+0000","last_change":"2023-12-14T18:24:36.589565+0000","last_active":"2023-12-14T18:24:41.941211+0000","last_peered":"2023-12-14T18:24:41.941211+0000","last_clean":"2023-12-14T18:24:41.941211+0000","last_became_active":"2023-12-14T18:24:36.588822+0000","last_became_peered":"2023-12-14T18:24:36.588822+0000","last_unstale":"2023-12-14T18:24:41.941211+0000","last_undegraded":"2023-12-14T18:24:41.941211+0000","last_fullsized":"2023-12-14T18:24:41.941211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:20:30.545830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941183+0000","last_change":"2023-12-14T18:24:36.593208+0000","last_active":"2023-12-14T18:24:41.941183+0000","last_peered":"2023-12-14T18:24:41.941183+0000","last_clean":"2023-12-14T18:24:41.941183+0000","last_became_active":"2023-12-14T18:24:36.586394+0000","last_became_peered":"2023-12-14T18:24:36.586394+0000","last_unstale":"2023-12-14T18:24:41.941183+0000","last_undegraded":"2023-12-14T18:24:41.941183+0000","last_fullsized":"2023-12-14T18:24:41.941183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:19:50.115165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940763+0000","last_change":"2023-12-14T18:24:36.594254+0000","last_active":"2023-12-14T18:24:41.940763+0000","last_peered":"2023-12-14T18:24:41.940763+0000","last_clean":"2023-12-14T18:24:41.940763+0000","last_became_active":"2023-12-14T18:24:36.593889+0000","last_became_peered":"2023-12-14T18:24:36.593889+0000","last_unstale":"2023-12-14T18:24:41.940763+0000","last_undegraded":"2023-12-14T18:24:41.940763+0000","last_fullsized":"2023-12-14T18:24:41.940763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:50.487280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939911+0000","last_change":"2023-12-14T18:24:36.584971+0000","last_active":"2023-12-14T18:24:41.939911+0000","last_peered":"2023-12-14T18:24:41.939911+0000","last_clean":"2023-12-14T18:24:41.939911+0000","last_became_active":"2023-12-14T18:24:36.584695+0000","last_became_peered":"2023-12-14T18:24:36.584695+0000","last_unstale":"2023-12-14T18:24:41.939911+0000","last_undegraded":"2023-12-14T18:24:41.939911+0000","last_fullsized":"2023-12-14T18:24:41.939911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:43:12.153321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939181+0000","last_change":"2023-12-14T18:24:36.589692+0000","last_active":"2023-12-14T18:24:41.939181+0000","last_peered":"2023-12-14T18:24:41.939181+0000","last_clean":"2023-12-14T18:24:41.939181+0000","last_became_active":"2023-12-14T18:24:36.589415+0000","last_became_peered":"2023-12-14T18:24:36.589415+0000","last_unstale":"2023-12-14T18:24:41.939181+0000","last_undegraded":"2023-12-14T18:24:41.939181+0000","last_fullsized":"2023-12-14T18:24:41.939181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:17:55.715589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.938924+0000","last_change":"2023-12-14T18:24:36.586911+0000","last_active":"2023-12-14T18:24:41.938924+0000","last_peered":"2023-12-14T18:24:41.938924+0000","last_clean":"2023-12-14T18:24:41.938924+0000","last_became_active":"2023-12-14T18:24:36.586600+0000","last_became_peered":"2023-12-14T18:24:36.586600+0000","last_unstale":"2023-12-14T18:24:41.938924+0000","last_undegraded":"2023-12-14T18:24:41.938924+0000","last_fullsized":"2023-12-14T18:24:41.938924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:00:56.150982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000169+0000","last_change":"2023-12-14T18:24:36.584879+0000","last_active":"2023-12-14T18:24:41.000169+0000","last_peered":"2023-12-14T18:24:41.000169+0000","last_clean":"2023-12-14T18:24:41.000169+0000","last_became_active":"2023-12-14T18:24:36.584566+0000","last_became_peered":"2023-12-14T18:24:36.584566+0000","last_unstale":"2023-12-14T18:24:41.000169+0000","last_undegraded":"2023-12-14T18:24:41.000169+0000","last_fullsized":"2023-12-14T18:24:41.000169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:23:00.946127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939646+0000","last_change":"2023-12-14T18:24:36.587420+0000","last_active":"2023-12-14T18:24:41.939646+0000","last_peered":"2023-12-14T18:24:41.939646+0000","last_clean":"2023-12-14T18:24:41.939646+0000","last_became_active":"2023-12-14T18:24:36.586732+0000","last_became_peered":"2023-12-14T18:24:36.586732+0000","last_unstale":"2023-12-14T18:24:41.939646+0000","last_undegraded":"2023-12-14T18:24:41.939646+0000","last_fullsized":"2023-12-14T18:24:41.939646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:06:51.142371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093222+0000","last_change":"2023-12-14T18:24:36.583412+0000","last_active":"2023-12-14T18:24:41.093222+0000","last_peered":"2023-12-14T18:24:41.093222+0000","last_clean":"2023-12-14T18:24:41.093222+0000","last_became_active":"2023-12-14T18:24:36.583022+0000","last_became_peered":"2023-12-14T18:24:36.583022+0000","last_unstale":"2023-12-14T18:24:41.093222+0000","last_undegraded":"2023-12-14T18:24:41.093222+0000","last_fullsized":"2023-12-14T18:24:41.093222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:35:26.281488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001176+0000","last_change":"2023-12-14T18:24:36.575232+0000","last_active":"2023-12-14T18:24:41.001176+0000","last_peered":"2023-12-14T18:24:41.001176+0000","last_clean":"2023-12-14T18:24:41.001176+0000","last_became_active":"2023-12-14T18:24:36.574573+0000","last_became_peered":"2023-12-14T18:24:36.574573+0000","last_unstale":"2023-12-14T18:24:41.001176+0000","last_undegraded":"2023-12-14T18:24:41.001176+0000","last_fullsized":"2023-12-14T18:24:41.001176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:09:44.789329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000573+0000","last_change":"2023-12-14T18:24:40.605077+0000","last_active":"2023-12-14T18:24:41.000573+0000","last_peered":"2023-12-14T18:24:41.000573+0000","last_clean":"2023-12-14T18:24:41.000573+0000","last_became_active":"2023-12-14T18:24:40.604646+0000","last_became_peered":"2023-12-14T18:24:40.604646+0000","last_unstale":"2023-12-14T18:24:41.000573+0000","last_undegraded":"2023-12-14T18:24:41.000573+0000","last_fullsized":"2023-12-14T18:24:41.000573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:46:49.342657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.768860+0000","last_change":"2023-12-14T18:24:36.177449+0000","last_active":"2023-12-14T18:25:00.768860+0000","last_peered":"2023-12-14T18:25:00.768860+0000","last_clean":"2023-12-14T18:25:00.768860+0000","last_became_active":"2023-12-14T18:24:36.177114+0000","last_became_peered":"2023-12-14T18:24:36.177114+0000","last_unstale":"2023-12-14T18:25:00.768860+0000","last_undegraded":"2023-12-14T18:25:00.768860+0000","last_fullsized":"2023-12-14T18:25:00.768860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:30.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999464+0000","last_change":"2023-12-14T18:24:36.574204+0000","last_active":"2023-12-14T18:24:40.999464+0000","last_peered":"2023-12-14T18:24:40.999464+0000","last_clean":"2023-12-14T18:24:40.999464+0000","last_became_active":"2023-12-14T18:24:36.573923+0000","last_became_peered":"2023-12-14T18:24:36.573923+0000","last_unstale":"2023-12-14T18:24:40.999464+0000","last_undegraded":"2023-12-14T18:24:40.999464+0000","last_fullsized":"2023-12-14T18:24:40.999464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:20:32.637706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939617+0000","last_change":"2023-12-14T18:24:40.600254+0000","last_active":"2023-12-14T18:24:41.939617+0000","last_peered":"2023-12-14T18:24:41.939617+0000","last_clean":"2023-12-14T18:24:41.939617+0000","last_became_active":"2023-12-14T18:24:40.599894+0000","last_became_peered":"2023-12-14T18:24:40.599894+0000","last_unstale":"2023-12-14T18:24:41.939617+0000","last_undegraded":"2023-12-14T18:24:41.939617+0000","last_fullsized":"2023-12-14T18:24:41.939617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:00.116137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999521+0000","last_change":"2023-12-14T18:24:36.170109+0000","last_active":"2023-12-14T18:24:40.999521+0000","last_peered":"2023-12-14T18:24:40.999521+0000","last_clean":"2023-12-14T18:24:40.999521+0000","last_became_active":"2023-12-14T18:24:36.169629+0000","last_became_peered":"2023-12-14T18:24:36.169629+0000","last_unstale":"2023-12-14T18:24:40.999521+0000","last_undegraded":"2023-12-14T18:24:40.999521+0000","last_fullsized":"2023-12-14T18:24:40.999521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:41:16.457493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939537+0000","last_change":"2023-12-14T18:24:36.582757+0000","last_active":"2023-12-14T18:24:41.939537+0000","last_peered":"2023-12-14T18:24:41.939537+0000","last_clean":"2023-12-14T18:24:41.939537+0000","last_became_active":"2023-12-14T18:24:36.582457+0000","last_became_peered":"2023-12-14T18:24:36.582457+0000","last_unstale":"2023-12-14T18:24:41.939537+0000","last_undegraded":"2023-12-14T18:24:41.939537+0000","last_fullsized":"2023-12-14T18:24:41.939537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:29:05.124428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940384+0000","last_change":"2023-12-14T18:24:40.599695+0000","last_active":"2023-12-14T18:24:41.940384+0000","last_peered":"2023-12-14T18:24:41.940384+0000","last_clean":"2023-12-14T18:24:41.940384+0000","last_became_active":"2023-12-14T18:24:40.599239+0000","last_became_peered":"2023-12-14T18:24:40.599239+0000","last_unstale":"2023-12-14T18:24:41.940384+0000","last_undegraded":"2023-12-14T18:24:41.940384+0000","last_fullsized":"2023-12-14T18:24:41.940384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:40.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095870+0000","last_change":"2023-12-14T18:24:36.177946+0000","last_active":"2023-12-14T18:24:41.095870+0000","last_peered":"2023-12-14T18:24:41.095870+0000","last_clean":"2023-12-14T18:24:41.095870+0000","last_became_active":"2023-12-14T18:24:36.177480+0000","last_became_peered":"2023-12-14T18:24:36.177480+0000","last_unstale":"2023-12-14T18:24:41.095870+0000","last_undegraded":"2023-12-14T18:24:41.095870+0000","last_fullsized":"2023-12-14T18:24:41.095870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:42:02.447975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000436+0000","last_change":"2023-12-14T18:24:36.587951+0000","last_active":"2023-12-14T18:24:41.000436+0000","last_peered":"2023-12-14T18:24:41.000436+0000","last_clean":"2023-12-14T18:24:41.000436+0000","last_became_active":"2023-12-14T18:24:36.587539+0000","last_became_peered":"2023-12-14T18:24:36.587539+0000","last_unstale":"2023-12-14T18:24:41.000436+0000","last_undegraded":"2023-12-14T18:24:41.000436+0000","last_fullsized":"2023-12-14T18:24:41.000436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:53:03.285569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941747+0000","last_change":"2023-12-14T18:24:40.600008+0000","last_active":"2023-12-14T18:24:41.941747+0000","last_peered":"2023-12-14T18:24:41.941747+0000","last_clean":"2023-12-14T18:24:41.941747+0000","last_became_active":"2023-12-14T18:24:40.599596+0000","last_became_peered":"2023-12-14T18:24:40.599596+0000","last_unstale":"2023-12-14T18:24:41.941747+0000","last_undegraded":"2023-12-14T18:24:41.941747+0000","last_fullsized":"2023-12-14T18:24:41.941747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:10:10.770024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095629+0000","last_change":"2023-12-14T18:24:36.176569+0000","last_active":"2023-12-14T18:24:41.095629+0000","last_peered":"2023-12-14T18:24:41.095629+0000","last_clean":"2023-12-14T18:24:41.095629+0000","last_became_active":"2023-12-14T18:24:36.176043+0000","last_became_peered":"2023-12-14T18:24:36.176043+0000","last_unstale":"2023-12-14T18:24:41.095629+0000","last_undegraded":"2023-12-14T18:24:41.095629+0000","last_fullsized":"2023-12-14T18:24:41.095629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:10:44.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149101+0000","last_change":"2023-12-14T18:24:36.593761+0000","last_active":"2023-12-14T18:24:41.149101+0000","last_peered":"2023-12-14T18:24:41.149101+0000","last_clean":"2023-12-14T18:24:41.149101+0000","last_became_active":"2023-12-14T18:24:36.593452+0000","last_became_peered":"2023-12-14T18:24:36.593452+0000","last_unstale":"2023-12-14T18:24:41.149101+0000","last_undegraded":"2023-12-14T18:24:41.149101+0000","last_fullsized":"2023-12-14T18:24:41.149101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:56:34.794908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001069+0000","last_change":"2023-12-14T18:24:40.598064+0000","last_active":"2023-12-14T18:24:41.001069+0000","last_peered":"2023-12-14T18:24:41.001069+0000","last_clean":"2023-12-14T18:24:41.001069+0000","last_became_active":"2023-12-14T18:24:40.597672+0000","last_became_peered":"2023-12-14T18:24:40.597672+0000","last_unstale":"2023-12-14T18:24:41.001069+0000","last_undegraded":"2023-12-14T18:24:41.001069+0000","last_fullsized":"2023-12-14T18:24:41.001069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:41:54.138227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.760893+0000","last_change":"2023-12-14T18:24:36.176288+0000","last_active":"2023-12-14T18:25:00.760893+0000","last_peered":"2023-12-14T18:25:00.760893+0000","last_clean":"2023-12-14T18:25:00.760893+0000","last_became_active":"2023-12-14T18:24:36.175965+0000","last_became_peered":"2023-12-14T18:24:36.175965+0000","last_unstale":"2023-12-14T18:25:00.760893+0000","last_undegraded":"2023-12-14T18:25:00.760893+0000","last_fullsized":"2023-12-14T18:25:00.760893+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:00:50.888829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095492+0000","last_change":"2023-12-14T18:24:36.583687+0000","last_active":"2023-12-14T18:24:41.095492+0000","last_peered":"2023-12-14T18:24:41.095492+0000","last_clean":"2023-12-14T18:24:41.095492+0000","last_became_active":"2023-12-14T18:24:36.583047+0000","last_became_peered":"2023-12-14T18:24:36.583047+0000","last_unstale":"2023-12-14T18:24:41.095492+0000","last_undegraded":"2023-12-14T18:24:41.095492+0000","last_fullsized":"2023-12-14T18:24:41.095492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:26:10.095059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001102+0000","last_change":"2023-12-14T18:24:40.605217+0000","last_active":"2023-12-14T18:24:41.001102+0000","last_peered":"2023-12-14T18:24:41.001102+0000","last_clean":"2023-12-14T18:24:41.001102+0000","last_became_active":"2023-12-14T18:24:40.604807+0000","last_became_peered":"2023-12-14T18:24:40.604807+0000","last_unstale":"2023-12-14T18:24:41.001102+0000","last_undegraded":"2023-12-14T18:24:41.001102+0000","last_fullsized":"2023-12-14T18:24:41.001102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:48:27.699822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767627+0000","last_change":"2023-12-14T18:24:36.174875+0000","last_active":"2023-12-14T18:25:00.767627+0000","last_peered":"2023-12-14T18:25:00.767627+0000","last_clean":"2023-12-14T18:25:00.767627+0000","last_became_active":"2023-12-14T18:24:36.173984+0000","last_became_peered":"2023-12-14T18:24:36.173984+0000","last_unstale":"2023-12-14T18:25:00.767627+0000","last_undegraded":"2023-12-14T18:25:00.767627+0000","last_fullsized":"2023-12-14T18:25:00.767627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:24:27.753878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149194+0000","last_change":"2023-12-14T18:24:36.580563+0000","last_active":"2023-12-14T18:24:41.149194+0000","last_peered":"2023-12-14T18:24:41.149194+0000","last_clean":"2023-12-14T18:24:41.149194+0000","last_became_active":"2023-12-14T18:24:36.580196+0000","last_became_peered":"2023-12-14T18:24:36.580196+0000","last_unstale":"2023-12-14T18:24:41.149194+0000","last_undegraded":"2023-12-14T18:24:41.149194+0000","last_fullsized":"2023-12-14T18:24:41.149194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:44:38.053372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939735+0000","last_change":"2023-12-14T18:24:40.601223+0000","last_active":"2023-12-14T18:24:41.939735+0000","last_peered":"2023-12-14T18:24:41.939735+0000","last_clean":"2023-12-14T18:24:41.939735+0000","last_became_active":"2023-12-14T18:24:40.600795+0000","last_became_peered":"2023-12-14T18:24:40.600795+0000","last_unstale":"2023-12-14T18:24:41.939735+0000","last_undegraded":"2023-12-14T18:24:41.939735+0000","last_fullsized":"2023-12-14T18:24:41.939735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:27:49.251012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767406+0000","last_change":"2023-12-14T18:24:36.173600+0000","last_active":"2023-12-14T18:25:00.767406+0000","last_peered":"2023-12-14T18:25:00.767406+0000","last_clean":"2023-12-14T18:25:00.767406+0000","last_became_active":"2023-12-14T18:24:36.173272+0000","last_became_peered":"2023-12-14T18:24:36.173272+0000","last_unstale":"2023-12-14T18:25:00.767406+0000","last_undegraded":"2023-12-14T18:25:00.767406+0000","last_fullsized":"2023-12-14T18:25:00.767406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:23.589714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998814+0000","last_change":"2023-12-14T18:24:36.575060+0000","last_active":"2023-12-14T18:24:40.998814+0000","last_peered":"2023-12-14T18:24:40.998814+0000","last_clean":"2023-12-14T18:24:40.998814+0000","last_became_active":"2023-12-14T18:24:36.574749+0000","last_became_peered":"2023-12-14T18:24:36.574749+0000","last_unstale":"2023-12-14T18:24:40.998814+0000","last_undegraded":"2023-12-14T18:24:40.998814+0000","last_fullsized":"2023-12-14T18:24:40.998814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:44:44.163762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939489+0000","last_change":"2023-12-14T18:24:40.605431+0000","last_active":"2023-12-14T18:24:41.939489+0000","last_peered":"2023-12-14T18:24:41.939489+0000","last_clean":"2023-12-14T18:24:41.939489+0000","last_became_active":"2023-12-14T18:24:40.604951+0000","last_became_peered":"2023-12-14T18:24:40.604951+0000","last_unstale":"2023-12-14T18:24:41.939489+0000","last_undegraded":"2023-12-14T18:24:41.939489+0000","last_fullsized":"2023-12-14T18:24:41.939489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:46:36.552203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148653+0000","last_change":"2023-12-14T18:24:36.175493+0000","last_active":"2023-12-14T18:24:41.148653+0000","last_peered":"2023-12-14T18:24:41.148653+0000","last_clean":"2023-12-14T18:24:41.148653+0000","last_became_active":"2023-12-14T18:24:36.175205+0000","last_became_peered":"2023-12-14T18:24:36.175205+0000","last_unstale":"2023-12-14T18:24:41.148653+0000","last_undegraded":"2023-12-14T18:24:41.148653+0000","last_fullsized":"2023-12-14T18:24:41.148653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:52:40.611249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092822+0000","last_change":"2023-12-14T18:24:36.587218+0000","last_active":"2023-12-14T18:24:41.092822+0000","last_peered":"2023-12-14T18:24:41.092822+0000","last_clean":"2023-12-14T18:24:41.092822+0000","last_became_active":"2023-12-14T18:24:36.586910+0000","last_became_peered":"2023-12-14T18:24:36.586910+0000","last_unstale":"2023-12-14T18:24:41.092822+0000","last_undegraded":"2023-12-14T18:24:41.092822+0000","last_fullsized":"2023-12-14T18:24:41.092822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:04.691462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999151+0000","last_change":"2023-12-14T18:24:40.598409+0000","last_active":"2023-12-14T18:24:40.999151+0000","last_peered":"2023-12-14T18:24:40.999151+0000","last_clean":"2023-12-14T18:24:40.999151+0000","last_became_active":"2023-12-14T18:24:40.598011+0000","last_became_peered":"2023-12-14T18:24:40.598011+0000","last_unstale":"2023-12-14T18:24:40.999151+0000","last_undegraded":"2023-12-14T18:24:40.999151+0000","last_fullsized":"2023-12-14T18:24:40.999151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:30.946044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039152+0000","last_change":"2023-12-14T18:24:36.172277+0000","last_active":"2023-12-14T18:25:00.039152+0000","last_peered":"2023-12-14T18:25:00.039152+0000","last_clean":"2023-12-14T18:25:00.039152+0000","last_became_active":"2023-12-14T18:24:36.171907+0000","last_became_peered":"2023-12-14T18:24:36.171907+0000","last_unstale":"2023-12-14T18:25:00.039152+0000","last_undegraded":"2023-12-14T18:25:00.039152+0000","last_fullsized":"2023-12-14T18:25:00.039152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:45:12.002124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000922+0000","last_change":"2023-12-14T18:24:36.584783+0000","last_active":"2023-12-14T18:24:41.000922+0000","last_peered":"2023-12-14T18:24:41.000922+0000","last_clean":"2023-12-14T18:24:41.000922+0000","last_became_active":"2023-12-14T18:24:36.584424+0000","last_became_peered":"2023-12-14T18:24:36.584424+0000","last_unstale":"2023-12-14T18:24:41.000922+0000","last_undegraded":"2023-12-14T18:24:41.000922+0000","last_fullsized":"2023-12-14T18:24:41.000922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:13:46.074921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092851+0000","last_change":"2023-12-14T18:24:40.603176+0000","last_active":"2023-12-14T18:24:41.092851+0000","last_peered":"2023-12-14T18:24:41.092851+0000","last_clean":"2023-12-14T18:24:41.092851+0000","last_became_active":"2023-12-14T18:24:40.602868+0000","last_became_peered":"2023-12-14T18:24:40.602868+0000","last_unstale":"2023-12-14T18:24:41.092851+0000","last_undegraded":"2023-12-14T18:24:41.092851+0000","last_fullsized":"2023-12-14T18:24:41.092851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:02:23.455860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.037799+0000","last_change":"2023-12-14T18:24:36.177638+0000","last_active":"2023-12-14T18:25:00.037799+0000","last_peered":"2023-12-14T18:25:00.037799+0000","last_clean":"2023-12-14T18:25:00.037799+0000","last_became_active":"2023-12-14T18:24:36.177335+0000","last_became_peered":"2023-12-14T18:24:36.177335+0000","last_unstale":"2023-12-14T18:25:00.037799+0000","last_undegraded":"2023-12-14T18:25:00.037799+0000","last_fullsized":"2023-12-14T18:25:00.037799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:50:02.813317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939828+0000","last_change":"2023-12-14T18:24:36.577508+0000","last_active":"2023-12-14T18:24:41.939828+0000","last_peered":"2023-12-14T18:24:41.939828+0000","last_clean":"2023-12-14T18:24:41.939828+0000","last_became_active":"2023-12-14T18:24:36.577196+0000","last_became_peered":"2023-12-14T18:24:36.577196+0000","last_unstale":"2023-12-14T18:24:41.939828+0000","last_undegraded":"2023-12-14T18:24:41.939828+0000","last_fullsized":"2023-12-14T18:24:41.939828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:32:16.278415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941240+0000","last_change":"2023-12-14T18:24:40.594816+0000","last_active":"2023-12-14T18:24:41.941240+0000","last_peered":"2023-12-14T18:24:41.941240+0000","last_clean":"2023-12-14T18:24:41.941240+0000","last_became_active":"2023-12-14T18:24:40.594371+0000","last_became_peered":"2023-12-14T18:24:40.594371+0000","last_unstale":"2023-12-14T18:24:41.941240+0000","last_undegraded":"2023-12-14T18:24:41.941240+0000","last_fullsized":"2023-12-14T18:24:41.941240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:40:46.343094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039258+0000","last_change":"2023-12-14T18:24:36.174238+0000","last_active":"2023-12-14T18:25:00.039258+0000","last_peered":"2023-12-14T18:25:00.039258+0000","last_clean":"2023-12-14T18:25:00.039258+0000","last_became_active":"2023-12-14T18:24:36.173373+0000","last_became_peered":"2023-12-14T18:24:36.173373+0000","last_unstale":"2023-12-14T18:25:00.039258+0000","last_undegraded":"2023-12-14T18:25:00.039258+0000","last_fullsized":"2023-12-14T18:25:00.039258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:40:50.640090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000757+0000","last_change":"2023-12-14T18:24:36.588589+0000","last_active":"2023-12-14T18:24:41.000757+0000","last_peered":"2023-12-14T18:24:41.000757+0000","last_clean":"2023-12-14T18:24:41.000757+0000","last_became_active":"2023-12-14T18:24:36.588269+0000","last_became_peered":"2023-12-14T18:24:36.588269+0000","last_unstale":"2023-12-14T18:24:41.000757+0000","last_undegraded":"2023-12-14T18:24:41.000757+0000","last_fullsized":"2023-12-14T18:24:41.000757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:12.316505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092841+0000","last_change":"2023-12-14T18:24:40.601333+0000","last_active":"2023-12-14T18:24:41.092841+0000","last_peered":"2023-12-14T18:24:41.092841+0000","last_clean":"2023-12-14T18:24:41.092841+0000","last_became_active":"2023-12-14T18:24:40.600966+0000","last_became_peered":"2023-12-14T18:24:40.600966+0000","last_unstale":"2023-12-14T18:24:41.092841+0000","last_undegraded":"2023-12-14T18:24:41.092841+0000","last_fullsized":"2023-12-14T18:24:41.092841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:36.433276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'11","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.759760+0000","last_change":"2023-12-14T18:24:36.177299+0000","last_active":"2023-12-14T18:25:00.759760+0000","last_peered":"2023-12-14T18:25:00.759760+0000","last_clean":"2023-12-14T18:25:00.759760+0000","last_became_active":"2023-12-14T18:24:36.176971+0000","last_became_peered":"2023-12-14T18:24:36.176971+0000","last_unstale":"2023-12-14T18:25:00.759760+0000","last_undegraded":"2023-12-14T18:25:00.759760+0000","last_fullsized":"2023-12-14T18:25:00.759760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:02:46.083153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095663+0000","last_change":"2023-12-14T18:24:36.586315+0000","last_active":"2023-12-14T18:24:41.095663+0000","last_peered":"2023-12-14T18:24:41.095663+0000","last_clean":"2023-12-14T18:24:41.095663+0000","last_became_active":"2023-12-14T18:24:36.586000+0000","last_became_peered":"2023-12-14T18:24:36.586000+0000","last_unstale":"2023-12-14T18:24:41.095663+0000","last_undegraded":"2023-12-14T18:24:41.095663+0000","last_fullsized":"2023-12-14T18:24:41.095663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:19:55.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149539+0000","last_change":"2023-12-14T18:24:40.596973+0000","last_active":"2023-12-14T18:24:41.149539+0000","last_peered":"2023-12-14T18:24:41.149539+0000","last_clean":"2023-12-14T18:24:41.149539+0000","last_became_active":"2023-12-14T18:24:40.596355+0000","last_became_peered":"2023-12-14T18:24:40.596355+0000","last_unstale":"2023-12-14T18:24:41.149539+0000","last_undegraded":"2023-12-14T18:24:41.149539+0000","last_fullsized":"2023-12-14T18:24:41.149539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:19:02.715446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767785+0000","last_change":"2023-12-14T18:24:36.175287+0000","last_active":"2023-12-14T18:25:00.767785+0000","last_peered":"2023-12-14T18:25:00.767785+0000","last_clean":"2023-12-14T18:25:00.767785+0000","last_became_active":"2023-12-14T18:24:36.174952+0000","last_became_peered":"2023-12-14T18:24:36.174952+0000","last_unstale":"2023-12-14T18:25:00.767785+0000","last_undegraded":"2023-12-14T18:25:00.767785+0000","last_fullsized":"2023-12-14T18:25:00.767785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:44:43.661729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940588+0000","last_change":"2023-12-14T18:24:36.582314+0000","last_active":"2023-12-14T18:24:41.940588+0000","last_peered":"2023-12-14T18:24:41.940588+0000","last_clean":"2023-12-14T18:24:41.940588+0000","last_became_active":"2023-12-14T18:24:36.581963+0000","last_became_peered":"2023-12-14T18:24:36.581963+0000","last_unstale":"2023-12-14T18:24:41.940588+0000","last_undegraded":"2023-12-14T18:24:41.940588+0000","last_fullsized":"2023-12-14T18:24:41.940588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:02:38.477328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939741+0000","last_change":"2023-12-14T18:24:40.599979+0000","last_active":"2023-12-14T18:24:41.939741+0000","last_peered":"2023-12-14T18:24:41.939741+0000","last_clean":"2023-12-14T18:24:41.939741+0000","last_became_active":"2023-12-14T18:24:40.599258+0000","last_became_peered":"2023-12-14T18:24:40.599258+0000","last_unstale":"2023-12-14T18:24:41.939741+0000","last_undegraded":"2023-12-14T18:24:41.939741+0000","last_fullsized":"2023-12-14T18:24:41.939741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:06.783797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039434+0000","last_change":"2023-12-14T18:24:36.175768+0000","last_active":"2023-12-14T18:25:00.039434+0000","last_peered":"2023-12-14T18:25:00.039434+0000","last_clean":"2023-12-14T18:25:00.039434+0000","last_became_active":"2023-12-14T18:24:36.174973+0000","last_became_peered":"2023-12-14T18:24:36.174973+0000","last_unstale":"2023-12-14T18:25:00.039434+0000","last_undegraded":"2023-12-14T18:25:00.039434+0000","last_fullsized":"2023-12-14T18:25:00.039434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:54.191789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940662+0000","last_change":"2023-12-14T18:24:36.591856+0000","last_active":"2023-12-14T18:24:41.940662+0000","last_peered":"2023-12-14T18:24:41.940662+0000","last_clean":"2023-12-14T18:24:41.940662+0000","last_became_active":"2023-12-14T18:24:36.591554+0000","last_became_peered":"2023-12-14T18:24:36.591554+0000","last_unstale":"2023-12-14T18:24:41.940662+0000","last_undegraded":"2023-12-14T18:24:41.940662+0000","last_fullsized":"2023-12-14T18:24:41.940662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:31:39.565049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093250+0000","last_change":"2023-12-14T18:24:40.603794+0000","last_active":"2023-12-14T18:24:41.093250+0000","last_peered":"2023-12-14T18:24:41.093250+0000","last_clean":"2023-12-14T18:24:41.093250+0000","last_became_active":"2023-12-14T18:24:40.603311+0000","last_became_peered":"2023-12-14T18:24:40.603311+0000","last_unstale":"2023-12-14T18:24:41.093250+0000","last_undegraded":"2023-12-14T18:24:41.093250+0000","last_fullsized":"2023-12-14T18:24:41.093250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:11:14.354480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939789+0000","last_change":"2023-12-14T18:24:36.177411+0000","last_active":"2023-12-14T18:24:41.939789+0000","last_peered":"2023-12-14T18:24:41.939789+0000","last_clean":"2023-12-14T18:24:41.939789+0000","last_became_active":"2023-12-14T18:24:36.177047+0000","last_became_peered":"2023-12-14T18:24:36.177047+0000","last_unstale":"2023-12-14T18:24:41.939789+0000","last_undegraded":"2023-12-14T18:24:41.939789+0000","last_fullsized":"2023-12-14T18:24:41.939789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:58:22.871429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939900+0000","last_change":"2023-12-14T18:24:36.597335+0000","last_active":"2023-12-14T18:24:41.939900+0000","last_peered":"2023-12-14T18:24:41.939900+0000","last_clean":"2023-12-14T18:24:41.939900+0000","last_became_active":"2023-12-14T18:24:36.596900+0000","last_became_peered":"2023-12-14T18:24:36.596900+0000","last_unstale":"2023-12-14T18:24:41.939900+0000","last_undegraded":"2023-12-14T18:24:41.939900+0000","last_fullsized":"2023-12-14T18:24:41.939900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:07:51.506541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940190+0000","last_change":"2023-12-14T18:24:40.607042+0000","last_active":"2023-12-14T18:24:41.940190+0000","last_peered":"2023-12-14T18:24:41.940190+0000","last_clean":"2023-12-14T18:24:41.940190+0000","last_became_active":"2023-12-14T18:24:40.606563+0000","last_became_peered":"2023-12-14T18:24:40.606563+0000","last_unstale":"2023-12-14T18:24:41.940190+0000","last_undegraded":"2023-12-14T18:24:41.940190+0000","last_fullsized":"2023-12-14T18:24:41.940190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:31:29.839031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'10","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.038304+0000","last_change":"2023-12-14T18:24:36.171994+0000","last_active":"2023-12-14T18:25:00.038304+0000","last_peered":"2023-12-14T18:25:00.038304+0000","last_clean":"2023-12-14T18:25:00.038304+0000","last_became_active":"2023-12-14T18:24:36.171661+0000","last_became_peered":"2023-12-14T18:24:36.171661+0000","last_unstale":"2023-12-14T18:25:00.038304+0000","last_undegraded":"2023-12-14T18:25:00.038304+0000","last_fullsized":"2023-12-14T18:25:00.038304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:33:33.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148674+0000","last_change":"2023-12-14T18:24:36.577818+0000","last_active":"2023-12-14T18:24:41.148674+0000","last_peered":"2023-12-14T18:24:41.148674+0000","last_clean":"2023-12-14T18:24:41.148674+0000","last_became_active":"2023-12-14T18:24:36.577356+0000","last_became_peered":"2023-12-14T18:24:36.577356+0000","last_unstale":"2023-12-14T18:24:41.148674+0000","last_undegraded":"2023-12-14T18:24:41.148674+0000","last_fullsized":"2023-12-14T18:24:41.148674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:22:05.663769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148886+0000","last_change":"2023-12-14T18:24:40.601995+0000","last_active":"2023-12-14T18:24:41.148886+0000","last_peered":"2023-12-14T18:24:41.148886+0000","last_clean":"2023-12-14T18:24:41.148886+0000","last_became_active":"2023-12-14T18:24:40.601571+0000","last_became_peered":"2023-12-14T18:24:40.601571+0000","last_unstale":"2023-12-14T18:24:41.148886+0000","last_undegraded":"2023-12-14T18:24:41.148886+0000","last_fullsized":"2023-12-14T18:24:41.148886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:30:58.566301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095876+0000","last_change":"2023-12-14T18:24:36.168661+0000","last_active":"2023-12-14T18:24:41.095876+0000","last_peered":"2023-12-14T18:24:41.095876+0000","last_clean":"2023-12-14T18:24:41.095876+0000","last_became_active":"2023-12-14T18:24:36.168299+0000","last_became_peered":"2023-12-14T18:24:36.168299+0000","last_unstale":"2023-12-14T18:24:41.095876+0000","last_undegraded":"2023-12-14T18:24:41.095876+0000","last_fullsized":"2023-12-14T18:24:41.095876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:55.288902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939900+0000","last_change":"2023-12-14T18:24:36.578417+0000","last_active":"2023-12-14T18:24:41.939900+0000","last_peered":"2023-12-14T18:24:41.939900+0000","last_clean":"2023-12-14T18:24:41.939900+0000","last_became_active":"2023-12-14T18:24:36.578080+0000","last_became_peered":"2023-12-14T18:24:36.578080+0000","last_unstale":"2023-12-14T18:24:41.939900+0000","last_undegraded":"2023-12-14T18:24:41.939900+0000","last_fullsized":"2023-12-14T18:24:41.939900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:07:18.978037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095510+0000","last_change":"2023-12-14T18:24:40.771866+0000","last_active":"2023-12-14T18:24:41.095510+0000","last_peered":"2023-12-14T18:24:41.095510+0000","last_clean":"2023-12-14T18:24:41.095510+0000","last_became_active":"2023-12-14T18:24:40.771324+0000","last_became_peered":"2023-12-14T18:24:40.771324+0000","last_unstale":"2023-12-14T18:24:41.095510+0000","last_undegraded":"2023-12-14T18:24:41.095510+0000","last_fullsized":"2023-12-14T18:24:41.095510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:20:19.058538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093047+0000","last_change":"2023-12-14T18:24:36.170489+0000","last_active":"2023-12-14T18:24:41.093047+0000","last_peered":"2023-12-14T18:24:41.093047+0000","last_clean":"2023-12-14T18:24:41.093047+0000","last_became_active":"2023-12-14T18:24:36.170161+0000","last_became_peered":"2023-12-14T18:24:36.170161+0000","last_unstale":"2023-12-14T18:24:41.093047+0000","last_undegraded":"2023-12-14T18:24:41.093047+0000","last_fullsized":"2023-12-14T18:24:41.093047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:33:56.690387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940761+0000","last_change":"2023-12-14T18:24:36.591094+0000","last_active":"2023-12-14T18:24:41.940761+0000","last_peered":"2023-12-14T18:24:41.940761+0000","last_clean":"2023-12-14T18:24:41.940761+0000","last_became_active":"2023-12-14T18:24:36.590779+0000","last_became_peered":"2023-12-14T18:24:36.590779+0000","last_unstale":"2023-12-14T18:24:41.940761+0000","last_undegraded":"2023-12-14T18:24:41.940761+0000","last_fullsized":"2023-12-14T18:24:41.940761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:16:00.885373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940628+0000","last_change":"2023-12-14T18:24:40.604125+0000","last_active":"2023-12-14T18:24:41.940628+0000","last_peered":"2023-12-14T18:24:41.940628+0000","last_clean":"2023-12-14T18:24:41.940628+0000","last_became_active":"2023-12-14T18:24:40.603242+0000","last_became_peered":"2023-12-14T18:24:40.603242+0000","last_unstale":"2023-12-14T18:24:41.940628+0000","last_undegraded":"2023-12-14T18:24:41.940628+0000","last_fullsized":"2023-12-14T18:24:41.940628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:52:35.630493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940851+0000","last_change":"2023-12-14T18:24:36.177567+0000","last_active":"2023-12-14T18:24:41.940851+0000","last_peered":"2023-12-14T18:24:41.940851+0000","last_clean":"2023-12-14T18:24:41.940851+0000","last_became_active":"2023-12-14T18:24:36.176949+0000","last_became_peered":"2023-12-14T18:24:36.176949+0000","last_unstale":"2023-12-14T18:24:41.940851+0000","last_undegraded":"2023-12-14T18:24:41.940851+0000","last_fullsized":"2023-12-14T18:24:41.940851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:13:55.523229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940032+0000","last_change":"2023-12-14T18:24:36.594224+0000","last_active":"2023-12-14T18:24:41.940032+0000","last_peered":"2023-12-14T18:24:41.940032+0000","last_clean":"2023-12-14T18:24:41.940032+0000","last_became_active":"2023-12-14T18:24:36.593859+0000","last_became_peered":"2023-12-14T18:24:36.593859+0000","last_unstale":"2023-12-14T18:24:41.940032+0000","last_undegraded":"2023-12-14T18:24:41.940032+0000","last_fullsized":"2023-12-14T18:24:41.940032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:59:04.044943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940173+0000","last_change":"2023-12-14T18:24:40.771814+0000","last_active":"2023-12-14T18:24:41.940173+0000","last_peered":"2023-12-14T18:24:41.940173+0000","last_clean":"2023-12-14T18:24:41.940173+0000","last_became_active":"2023-12-14T18:24:40.771387+0000","last_became_peered":"2023-12-14T18:24:40.771387+0000","last_unstale":"2023-12-14T18:24:41.940173+0000","last_undegraded":"2023-12-14T18:24:41.940173+0000","last_fullsized":"2023-12-14T18:24:41.940173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:22.744059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148815+0000","last_change":"2023-12-14T18:24:36.588826+0000","last_active":"2023-12-14T18:24:41.148815+0000","last_peered":"2023-12-14T18:24:41.148815+0000","last_clean":"2023-12-14T18:24:41.148815+0000","last_became_active":"2023-12-14T18:24:36.588320+0000","last_became_peered":"2023-12-14T18:24:36.588320+0000","last_unstale":"2023-12-14T18:24:41.148815+0000","last_undegraded":"2023-12-14T18:24:41.148815+0000","last_fullsized":"2023-12-14T18:24:41.148815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:15:09.573489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.768534+0000","last_change":"2023-12-14T18:24:36.173995+0000","last_active":"2023-12-14T18:25:00.768534+0000","last_peered":"2023-12-14T18:25:00.768534+0000","last_clean":"2023-12-14T18:25:00.768534+0000","last_became_active":"2023-12-14T18:24:36.173381+0000","last_became_peered":"2023-12-14T18:24:36.173381+0000","last_unstale":"2023-12-14T18:25:00.768534+0000","last_undegraded":"2023-12-14T18:25:00.768534+0000","last_fullsized":"2023-12-14T18:25:00.768534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:17:33.452268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:42.862743+0000","last_change":"2023-12-14T18:24:40.605099+0000","last_active":"2023-12-14T18:24:42.862743+0000","last_peered":"2023-12-14T18:24:42.862743+0000","last_clean":"2023-12-14T18:24:42.862743+0000","last_became_active":"2023-12-14T18:24:40.604773+0000","last_became_peered":"2023-12-14T18:24:40.604773+0000","last_unstale":"2023-12-14T18:24:42.862743+0000","last_undegraded":"2023-12-14T18:24:42.862743+0000","last_fullsized":"2023-12-14T18:24:42.862743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:27:44.202301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998989+0000","last_change":"2023-12-14T18:24:36.574993+0000","last_active":"2023-12-14T18:24:40.998989+0000","last_peered":"2023-12-14T18:24:40.998989+0000","last_clean":"2023-12-14T18:24:40.998989+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:24:40.998989+0000","last_undegraded":"2023-12-14T18:24:40.998989+0000","last_fullsized":"2023-12-14T18:24:40.998989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:36:49.384865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039428+0000","last_change":"2023-12-14T18:24:36.170061+0000","last_active":"2023-12-14T18:25:00.039428+0000","last_peered":"2023-12-14T18:25:00.039428+0000","last_clean":"2023-12-14T18:25:00.039428+0000","last_became_active":"2023-12-14T18:24:36.169652+0000","last_became_peered":"2023-12-14T18:24:36.169652+0000","last_unstale":"2023-12-14T18:25:00.039428+0000","last_undegraded":"2023-12-14T18:25:00.039428+0000","last_fullsized":"2023-12-14T18:25:00.039428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:35:33.204173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941240+0000","last_change":"2023-12-14T18:23:49.076644+0000","last_active":"2023-12-14T18:24:41.941240+0000","last_peered":"2023-12-14T18:24:41.941240+0000","last_clean":"2023-12-14T18:24:41.941240+0000","last_became_active":"2023-12-14T18:23:49.076263+0000","last_became_peered":"2023-12-14T18:23:49.076263+0000","last_unstale":"2023-12-14T18:24:41.941240+0000","last_undegraded":"2023-12-14T18:24:41.941240+0000","last_fullsized":"2023-12-14T18:24:41.941240+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_clean_scrub_stamp":"2023-12-14T18:21:24.715131+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:19:15.132541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149430+0000","last_change":"2023-12-14T18:24:40.772015+0000","last_active":"2023-12-14T18:24:41.149430+0000","last_peered":"2023-12-14T18:24:41.149430+0000","last_clean":"2023-12-14T18:24:41.149430+0000","last_became_active":"2023-12-14T18:24:40.771497+0000","last_became_peered":"2023-12-14T18:24:40.771497+0000","last_unstale":"2023-12-14T18:24:41.149430+0000","last_undegraded":"2023-12-14T18:24:41.149430+0000","last_fullsized":"2023-12-14T18:24:41.149430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:37:04.368364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939552+0000","last_change":"2023-12-14T18:24:36.584402+0000","last_active":"2023-12-14T18:24:41.939552+0000","last_peered":"2023-12-14T18:24:41.939552+0000","last_clean":"2023-12-14T18:24:41.939552+0000","last_became_active":"2023-12-14T18:24:36.584177+0000","last_became_peered":"2023-12-14T18:24:36.584177+0000","last_unstale":"2023-12-14T18:24:41.939552+0000","last_undegraded":"2023-12-14T18:24:41.939552+0000","last_fullsized":"2023-12-14T18:24:41.939552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:38:45.449137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767235+0000","last_change":"2023-12-14T18:24:36.175017+0000","last_active":"2023-12-14T18:25:00.767235+0000","last_peered":"2023-12-14T18:25:00.767235+0000","last_clean":"2023-12-14T18:25:00.767235+0000","last_became_active":"2023-12-14T18:24:36.174639+0000","last_became_peered":"2023-12-14T18:24:36.174639+0000","last_unstale":"2023-12-14T18:25:00.767235+0000","last_undegraded":"2023-12-14T18:25:00.767235+0000","last_fullsized":"2023-12-14T18:25:00.767235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:56:02.780313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939585+0000","last_change":"2023-12-14T18:24:40.596043+0000","last_active":"2023-12-14T18:24:41.939585+0000","last_peered":"2023-12-14T18:24:41.939585+0000","last_clean":"2023-12-14T18:24:41.939585+0000","last_became_active":"2023-12-14T18:24:40.595603+0000","last_became_peered":"2023-12-14T18:24:40.595603+0000","last_unstale":"2023-12-14T18:24:41.939585+0000","last_undegraded":"2023-12-14T18:24:41.939585+0000","last_fullsized":"2023-12-14T18:24:41.939585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:09:47.437024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095858+0000","last_change":"2023-12-14T18:24:36.177484+0000","last_active":"2023-12-14T18:24:41.095858+0000","last_peered":"2023-12-14T18:24:41.095858+0000","last_clean":"2023-12-14T18:24:41.095858+0000","last_became_active":"2023-12-14T18:24:36.176592+0000","last_became_peered":"2023-12-14T18:24:36.176592+0000","last_unstale":"2023-12-14T18:24:41.095858+0000","last_undegraded":"2023-12-14T18:24:41.095858+0000","last_fullsized":"2023-12-14T18:24:41.095858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:21:55.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940747+0000","last_change":"2023-12-14T18:24:36.585687+0000","last_active":"2023-12-14T18:24:41.940747+0000","last_peered":"2023-12-14T18:24:41.940747+0000","last_clean":"2023-12-14T18:24:41.940747+0000","last_became_active":"2023-12-14T18:24:36.585360+0000","last_became_peered":"2023-12-14T18:24:36.585360+0000","last_unstale":"2023-12-14T18:24:41.940747+0000","last_undegraded":"2023-12-14T18:24:41.940747+0000","last_fullsized":"2023-12-14T18:24:41.940747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:42.737826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998830+0000","last_change":"2023-12-14T18:24:40.771747+0000","last_active":"2023-12-14T18:24:40.998830+0000","last_peered":"2023-12-14T18:24:40.998830+0000","last_clean":"2023-12-14T18:24:40.998830+0000","last_became_active":"2023-12-14T18:24:40.771310+0000","last_became_peered":"2023-12-14T18:24:40.771310+0000","last_unstale":"2023-12-14T18:24:40.998830+0000","last_undegraded":"2023-12-14T18:24:40.998830+0000","last_fullsized":"2023-12-14T18:24:40.998830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:52.843438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767966+0000","last_change":"2023-12-14T18:24:36.171833+0000","last_active":"2023-12-14T18:25:00.767966+0000","last_peered":"2023-12-14T18:25:00.767966+0000","last_clean":"2023-12-14T18:25:00.767966+0000","last_became_active":"2023-12-14T18:24:36.171512+0000","last_became_peered":"2023-12-14T18:24:36.171512+0000","last_unstale":"2023-12-14T18:25:00.767966+0000","last_undegraded":"2023-12-14T18:25:00.767966+0000","last_fullsized":"2023-12-14T18:25:00.767966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:56.553040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999452+0000","last_change":"2023-12-14T18:24:36.594938+0000","last_active":"2023-12-14T18:24:40.999452+0000","last_peered":"2023-12-14T18:24:40.999452+0000","last_clean":"2023-12-14T18:24:40.999452+0000","last_became_active":"2023-12-14T18:24:36.594573+0000","last_became_peered":"2023-12-14T18:24:36.594573+0000","last_unstale":"2023-12-14T18:24:40.999452+0000","last_undegraded":"2023-12-14T18:24:40.999452+0000","last_fullsized":"2023-12-14T18:24:40.999452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:18:30.843187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000512+0000","last_change":"2023-12-14T18:24:40.601146+0000","last_active":"2023-12-14T18:24:41.000512+0000","last_peered":"2023-12-14T18:24:41.000512+0000","last_clean":"2023-12-14T18:24:41.000512+0000","last_became_active":"2023-12-14T18:24:40.600745+0000","last_became_peered":"2023-12-14T18:24:40.600745+0000","last_unstale":"2023-12-14T18:24:41.000512+0000","last_undegraded":"2023-12-14T18:24:41.000512+0000","last_fullsized":"2023-12-14T18:24:41.000512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:48:20.003480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039339+0000","last_change":"2023-12-14T18:24:36.175880+0000","last_active":"2023-12-14T18:25:00.039339+0000","last_peered":"2023-12-14T18:25:00.039339+0000","last_clean":"2023-12-14T18:25:00.039339+0000","last_became_active":"2023-12-14T18:24:36.175514+0000","last_became_peered":"2023-12-14T18:24:36.175514+0000","last_unstale":"2023-12-14T18:25:00.039339+0000","last_undegraded":"2023-12-14T18:25:00.039339+0000","last_fullsized":"2023-12-14T18:25:00.039339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:00:40.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939608+0000","last_change":"2023-12-14T18:24:36.577062+0000","last_active":"2023-12-14T18:24:41.939608+0000","last_peered":"2023-12-14T18:24:41.939608+0000","last_clean":"2023-12-14T18:24:41.939608+0000","last_became_active":"2023-12-14T18:24:36.576733+0000","last_became_peered":"2023-12-14T18:24:36.576733+0000","last_unstale":"2023-12-14T18:24:41.939608+0000","last_undegraded":"2023-12-14T18:24:41.939608+0000","last_fullsized":"2023-12-14T18:24:41.939608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:12:08.207474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940397+0000","last_change":"2023-12-14T18:24:40.772752+0000","last_active":"2023-12-14T18:24:41.940397+0000","last_peered":"2023-12-14T18:24:41.940397+0000","last_clean":"2023-12-14T18:24:41.940397+0000","last_became_active":"2023-12-14T18:24:40.772168+0000","last_became_peered":"2023-12-14T18:24:40.772168+0000","last_unstale":"2023-12-14T18:24:41.940397+0000","last_undegraded":"2023-12-14T18:24:41.940397+0000","last_fullsized":"2023-12-14T18:24:41.940397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:02:24.129455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.038773+0000","last_change":"2023-12-14T18:24:36.178885+0000","last_active":"2023-12-14T18:25:00.038773+0000","last_peered":"2023-12-14T18:25:00.038773+0000","last_clean":"2023-12-14T18:25:00.038773+0000","last_became_active":"2023-12-14T18:24:36.178386+0000","last_became_peered":"2023-12-14T18:24:36.178386+0000","last_unstale":"2023-12-14T18:25:00.038773+0000","last_undegraded":"2023-12-14T18:25:00.038773+0000","last_fullsized":"2023-12-14T18:25:00.038773+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:37:16.134801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095247+0000","last_change":"2023-12-14T18:24:36.590037+0000","last_active":"2023-12-14T18:24:41.095247+0000","last_peered":"2023-12-14T18:24:41.095247+0000","last_clean":"2023-12-14T18:24:41.095247+0000","last_became_active":"2023-12-14T18:24:36.589659+0000","last_became_peered":"2023-12-14T18:24:36.589659+0000","last_unstale":"2023-12-14T18:24:41.095247+0000","last_undegraded":"2023-12-14T18:24:41.095247+0000","last_fullsized":"2023-12-14T18:24:41.095247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:13:57.949133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093418+0000","last_change":"2023-12-14T18:24:40.602483+0000","last_active":"2023-12-14T18:24:41.093418+0000","last_peered":"2023-12-14T18:24:41.093418+0000","last_clean":"2023-12-14T18:24:41.093418+0000","last_became_active":"2023-12-14T18:24:40.601989+0000","last_became_peered":"2023-12-14T18:24:40.601989+0000","last_unstale":"2023-12-14T18:24:41.093418+0000","last_undegraded":"2023-12-14T18:24:41.093418+0000","last_fullsized":"2023-12-14T18:24:41.093418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:09.202695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.037079+0000","last_change":"2023-12-14T18:24:36.173350+0000","last_active":"2023-12-14T18:25:00.037079+0000","last_peered":"2023-12-14T18:25:00.037079+0000","last_clean":"2023-12-14T18:25:00.037079+0000","last_became_active":"2023-12-14T18:24:36.172840+0000","last_became_peered":"2023-12-14T18:24:36.172840+0000","last_unstale":"2023-12-14T18:25:00.037079+0000","last_undegraded":"2023-12-14T18:25:00.037079+0000","last_fullsized":"2023-12-14T18:25:00.037079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:17:31.439315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939457+0000","last_change":"2023-12-14T18:24:36.592104+0000","last_active":"2023-12-14T18:24:41.939457+0000","last_peered":"2023-12-14T18:24:41.939457+0000","last_clean":"2023-12-14T18:24:41.939457+0000","last_became_active":"2023-12-14T18:24:36.591568+0000","last_became_peered":"2023-12-14T18:24:36.591568+0000","last_unstale":"2023-12-14T18:24:41.939457+0000","last_undegraded":"2023-12-14T18:24:41.939457+0000","last_fullsized":"2023-12-14T18:24:41.939457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:30.952113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940520+0000","last_change":"2023-12-14T18:24:40.596043+0000","last_active":"2023-12-14T18:24:41.940520+0000","last_peered":"2023-12-14T18:24:41.940520+0000","last_clean":"2023-12-14T18:24:41.940520+0000","last_became_active":"2023-12-14T18:24:40.595520+0000","last_became_peered":"2023-12-14T18:24:40.595520+0000","last_unstale":"2023-12-14T18:24:41.940520+0000","last_undegraded":"2023-12-14T18:24:41.940520+0000","last_fullsized":"2023-12-14T18:24:41.940520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:45:44.615227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000728+0000","last_change":"2023-12-14T18:24:36.174184+0000","last_active":"2023-12-14T18:24:41.000728+0000","last_peered":"2023-12-14T18:24:41.000728+0000","last_clean":"2023-12-14T18:24:41.000728+0000","last_became_active":"2023-12-14T18:24:36.173850+0000","last_became_peered":"2023-12-14T18:24:36.173850+0000","last_unstale":"2023-12-14T18:24:41.000728+0000","last_undegraded":"2023-12-14T18:24:41.000728+0000","last_fullsized":"2023-12-14T18:24:41.000728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:49:58.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999206+0000","last_change":"2023-12-14T18:24:36.587777+0000","last_active":"2023-12-14T18:24:40.999206+0000","last_peered":"2023-12-14T18:24:40.999206+0000","last_clean":"2023-12-14T18:24:40.999206+0000","last_became_active":"2023-12-14T18:24:36.587317+0000","last_became_peered":"2023-12-14T18:24:36.587317+0000","last_unstale":"2023-12-14T18:24:40.999206+0000","last_undegraded":"2023-12-14T18:24:40.999206+0000","last_fullsized":"2023-12-14T18:24:40.999206+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:59:49.931215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940400+0000","last_change":"2023-12-14T18:24:40.771913+0000","last_active":"2023-12-14T18:24:41.940400+0000","last_peered":"2023-12-14T18:24:41.940400+0000","last_clean":"2023-12-14T18:24:41.940400+0000","last_became_active":"2023-12-14T18:24:40.771425+0000","last_became_peered":"2023-12-14T18:24:40.771425+0000","last_unstale":"2023-12-14T18:24:41.940400+0000","last_undegraded":"2023-12-14T18:24:41.940400+0000","last_fullsized":"2023-12-14T18:24:41.940400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:17:43.378002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039322+0000","last_change":"2023-12-14T18:24:36.176801+0000","last_active":"2023-12-14T18:25:00.039322+0000","last_peered":"2023-12-14T18:25:00.039322+0000","last_clean":"2023-12-14T18:25:00.039322+0000","last_became_active":"2023-12-14T18:24:36.176001+0000","last_became_peered":"2023-12-14T18:24:36.176001+0000","last_unstale":"2023-12-14T18:25:00.039322+0000","last_undegraded":"2023-12-14T18:25:00.039322+0000","last_fullsized":"2023-12-14T18:25:00.039322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:16:31.583481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940941+0000","last_change":"2023-12-14T18:24:36.592898+0000","last_active":"2023-12-14T18:24:41.940941+0000","last_peered":"2023-12-14T18:24:41.940941+0000","last_clean":"2023-12-14T18:24:41.940941+0000","last_became_active":"2023-12-14T18:24:36.586650+0000","last_became_peered":"2023-12-14T18:24:36.586650+0000","last_unstale":"2023-12-14T18:24:41.940941+0000","last_undegraded":"2023-12-14T18:24:41.940941+0000","last_fullsized":"2023-12-14T18:24:41.940941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:39:25.660709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000369+0000","last_change":"2023-12-14T18:24:40.599181+0000","last_active":"2023-12-14T18:24:41.000369+0000","last_peered":"2023-12-14T18:24:41.000369+0000","last_clean":"2023-12-14T18:24:41.000369+0000","last_became_active":"2023-12-14T18:24:40.598785+0000","last_became_peered":"2023-12-14T18:24:40.598785+0000","last_unstale":"2023-12-14T18:24:41.000369+0000","last_undegraded":"2023-12-14T18:24:41.000369+0000","last_fullsized":"2023-12-14T18:24:41.000369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:11:55.342182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767511+0000","last_change":"2023-12-14T18:24:36.173644+0000","last_active":"2023-12-14T18:25:00.767511+0000","last_peered":"2023-12-14T18:25:00.767511+0000","last_clean":"2023-12-14T18:25:00.767511+0000","last_became_active":"2023-12-14T18:24:36.173059+0000","last_became_peered":"2023-12-14T18:24:36.173059+0000","last_unstale":"2023-12-14T18:25:00.767511+0000","last_undegraded":"2023-12-14T18:25:00.767511+0000","last_fullsized":"2023-12-14T18:25:00.767511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:11:21.509881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941424+0000","last_change":"2023-12-14T18:24:36.588761+0000","last_active":"2023-12-14T18:24:41.941424+0000","last_peered":"2023-12-14T18:24:41.941424+0000","last_clean":"2023-12-14T18:24:41.941424+0000","last_became_active":"2023-12-14T18:24:36.586021+0000","last_became_peered":"2023-12-14T18:24:36.586021+0000","last_unstale":"2023-12-14T18:24:41.941424+0000","last_undegraded":"2023-12-14T18:24:41.941424+0000","last_fullsized":"2023-12-14T18:24:41.941424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:23:36.358426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939203+0000","last_change":"2023-12-14T18:24:40.607286+0000","last_active":"2023-12-14T18:24:41.939203+0000","last_peered":"2023-12-14T18:24:41.939203+0000","last_clean":"2023-12-14T18:24:41.939203+0000","last_became_active":"2023-12-14T18:24:40.606925+0000","last_became_peered":"2023-12-14T18:24:40.606925+0000","last_unstale":"2023-12-14T18:24:41.939203+0000","last_undegraded":"2023-12-14T18:24:41.939203+0000","last_fullsized":"2023-12-14T18:24:41.939203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:47:39.360084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149531+0000","last_change":"2023-12-14T18:24:36.174467+0000","last_active":"2023-12-14T18:24:41.149531+0000","last_peered":"2023-12-14T18:24:41.149531+0000","last_clean":"2023-12-14T18:24:41.149531+0000","last_became_active":"2023-12-14T18:24:36.173920+0000","last_became_peered":"2023-12-14T18:24:36.173920+0000","last_unstale":"2023-12-14T18:24:41.149531+0000","last_undegraded":"2023-12-14T18:24:41.149531+0000","last_fullsized":"2023-12-14T18:24:41.149531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:36:25.392784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940237+0000","last_change":"2023-12-14T18:24:36.597537+0000","last_active":"2023-12-14T18:24:41.940237+0000","last_peered":"2023-12-14T18:24:41.940237+0000","last_clean":"2023-12-14T18:24:41.940237+0000","last_became_active":"2023-12-14T18:24:36.594599+0000","last_became_peered":"2023-12-14T18:24:36.594599+0000","last_unstale":"2023-12-14T18:24:41.940237+0000","last_undegraded":"2023-12-14T18:24:41.940237+0000","last_fullsized":"2023-12-14T18:24:41.940237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:00.181838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148562+0000","last_change":"2023-12-14T18:24:40.597990+0000","last_active":"2023-12-14T18:24:41.148562+0000","last_peered":"2023-12-14T18:24:41.148562+0000","last_clean":"2023-12-14T18:24:41.148562+0000","last_became_active":"2023-12-14T18:24:40.597568+0000","last_became_peered":"2023-12-14T18:24:40.597568+0000","last_unstale":"2023-12-14T18:24:41.148562+0000","last_undegraded":"2023-12-14T18:24:41.148562+0000","last_fullsized":"2023-12-14T18:24:41.148562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:31:38.963162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940025+0000","last_change":"2023-12-14T18:24:36.173141+0000","last_active":"2023-12-14T18:24:41.940025+0000","last_peered":"2023-12-14T18:24:41.940025+0000","last_clean":"2023-12-14T18:24:41.940025+0000","last_became_active":"2023-12-14T18:24:36.172811+0000","last_became_peered":"2023-12-14T18:24:36.172811+0000","last_unstale":"2023-12-14T18:24:41.940025+0000","last_undegraded":"2023-12-14T18:24:41.940025+0000","last_fullsized":"2023-12-14T18:24:41.940025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:01:41.375107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941679+0000","last_change":"2023-12-14T18:24:36.587333+0000","last_active":"2023-12-14T18:24:41.941679+0000","last_peered":"2023-12-14T18:24:41.941679+0000","last_clean":"2023-12-14T18:24:41.941679+0000","last_became_active":"2023-12-14T18:24:36.586914+0000","last_became_peered":"2023-12-14T18:24:36.586914+0000","last_unstale":"2023-12-14T18:24:41.941679+0000","last_undegraded":"2023-12-14T18:24:41.941679+0000","last_fullsized":"2023-12-14T18:24:41.941679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:44:34.232245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939625+0000","last_change":"2023-12-14T18:24:36.591896+0000","last_active":"2023-12-14T18:24:41.939625+0000","last_peered":"2023-12-14T18:24:41.939625+0000","last_clean":"2023-12-14T18:24:41.939625+0000","last_became_active":"2023-12-14T18:24:36.591097+0000","last_became_peered":"2023-12-14T18:24:36.591097+0000","last_unstale":"2023-12-14T18:24:41.939625+0000","last_undegraded":"2023-12-14T18:24:41.939625+0000","last_fullsized":"2023-12-14T18:24:41.939625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:14:56.435755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000778+0000","last_change":"2023-12-14T18:24:36.581688+0000","last_active":"2023-12-14T18:24:41.000778+0000","last_peered":"2023-12-14T18:24:41.000778+0000","last_clean":"2023-12-14T18:24:41.000778+0000","last_became_active":"2023-12-14T18:24:36.580660+0000","last_became_peered":"2023-12-14T18:24:36.580660+0000","last_unstale":"2023-12-14T18:24:41.000778+0000","last_undegraded":"2023-12-14T18:24:41.000778+0000","last_fullsized":"2023-12-14T18:24:41.000778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:13:48.827412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000650+0000","last_change":"2023-12-14T18:24:36.583025+0000","last_active":"2023-12-14T18:24:41.000650+0000","last_peered":"2023-12-14T18:24:41.000650+0000","last_clean":"2023-12-14T18:24:41.000650+0000","last_became_active":"2023-12-14T18:24:36.582510+0000","last_became_peered":"2023-12-14T18:24:36.582510+0000","last_unstale":"2023-12-14T18:24:41.000650+0000","last_undegraded":"2023-12-14T18:24:41.000650+0000","last_fullsized":"2023-12-14T18:24:41.000650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:21:38.503753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939373+0000","last_change":"2023-12-14T18:24:36.579383+0000","last_active":"2023-12-14T18:24:41.939373+0000","last_peered":"2023-12-14T18:24:41.939373+0000","last_clean":"2023-12-14T18:24:41.939373+0000","last_became_active":"2023-12-14T18:24:36.579037+0000","last_became_peered":"2023-12-14T18:24:36.579037+0000","last_unstale":"2023-12-14T18:24:41.939373+0000","last_undegraded":"2023-12-14T18:24:41.939373+0000","last_fullsized":"2023-12-14T18:24:41.939373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:13:32.810036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940749+0000","last_change":"2023-12-14T18:24:36.582940+0000","last_active":"2023-12-14T18:24:41.940749+0000","last_peered":"2023-12-14T18:24:41.940749+0000","last_clean":"2023-12-14T18:24:41.940749+0000","last_became_active":"2023-12-14T18:24:36.582645+0000","last_became_peered":"2023-12-14T18:24:36.582645+0000","last_unstale":"2023-12-14T18:24:41.940749+0000","last_undegraded":"2023-12-14T18:24:41.940749+0000","last_fullsized":"2023-12-14T18:24:41.940749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:43:46.761797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940015+0000","last_change":"2023-12-14T18:24:36.578613+0000","last_active":"2023-12-14T18:24:41.940015+0000","last_peered":"2023-12-14T18:24:41.940015+0000","last_clean":"2023-12-14T18:24:41.940015+0000","last_became_active":"2023-12-14T18:24:36.578287+0000","last_became_peered":"2023-12-14T18:24:36.578287+0000","last_unstale":"2023-12-14T18:24:41.940015+0000","last_undegraded":"2023-12-14T18:24:41.940015+0000","last_fullsized":"2023-12-14T18:24:41.940015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:05:43.072668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001258+0000","last_change":"2023-12-14T18:24:36.575078+0000","last_active":"2023-12-14T18:24:41.001258+0000","last_peered":"2023-12-14T18:24:41.001258+0000","last_clean":"2023-12-14T18:24:41.001258+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:24:41.001258+0000","last_undegraded":"2023-12-14T18:24:41.001258+0000","last_fullsized":"2023-12-14T18:24:41.001258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:55:31.886500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999571+0000","last_change":"2023-12-14T18:24:36.572545+0000","last_active":"2023-12-14T18:24:40.999571+0000","last_peered":"2023-12-14T18:24:40.999571+0000","last_clean":"2023-12-14T18:24:40.999571+0000","last_became_active":"2023-12-14T18:24:36.572235+0000","last_became_peered":"2023-12-14T18:24:36.572235+0000","last_unstale":"2023-12-14T18:24:40.999571+0000","last_undegraded":"2023-12-14T18:24:40.999571+0000","last_fullsized":"2023-12-14T18:24:40.999571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:25:56.081431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148685+0000","last_change":"2023-12-14T18:24:36.588278+0000","last_active":"2023-12-14T18:24:41.148685+0000","last_peered":"2023-12-14T18:24:41.148685+0000","last_clean":"2023-12-14T18:24:41.148685+0000","last_became_active":"2023-12-14T18:24:36.587605+0000","last_became_peered":"2023-12-14T18:24:36.587605+0000","last_unstale":"2023-12-14T18:24:41.148685+0000","last_undegraded":"2023-12-14T18:24:41.148685+0000","last_fullsized":"2023-12-14T18:24:41.148685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:32:11.935687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998910+0000","last_change":"2023-12-14T18:24:36.575325+0000","last_active":"2023-12-14T18:24:40.998910+0000","last_peered":"2023-12-14T18:24:40.998910+0000","last_clean":"2023-12-14T18:24:40.998910+0000","last_became_active":"2023-12-14T18:24:36.575016+0000","last_became_peered":"2023-12-14T18:24:36.575016+0000","last_unstale":"2023-12-14T18:24:40.998910+0000","last_undegraded":"2023-12-14T18:24:40.998910+0000","last_fullsized":"2023-12-14T18:24:40.998910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:35:26.394701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000615+0000","last_change":"2023-12-14T18:24:36.585348+0000","last_active":"2023-12-14T18:24:41.000615+0000","last_peered":"2023-12-14T18:24:41.000615+0000","last_clean":"2023-12-14T18:24:41.000615+0000","last_became_active":"2023-12-14T18:24:36.584995+0000","last_became_peered":"2023-12-14T18:24:36.584995+0000","last_unstale":"2023-12-14T18:24:41.000615+0000","last_undegraded":"2023-12-14T18:24:41.000615+0000","last_fullsized":"2023-12-14T18:24:41.000615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:22:10.788069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939979+0000","last_change":"2023-12-14T18:24:36.583571+0000","last_active":"2023-12-14T18:24:41.939979+0000","last_peered":"2023-12-14T18:24:41.939979+0000","last_clean":"2023-12-14T18:24:41.939979+0000","last_became_active":"2023-12-14T18:24:36.583266+0000","last_became_peered":"2023-12-14T18:24:36.583266+0000","last_unstale":"2023-12-14T18:24:41.939979+0000","last_undegraded":"2023-12-14T18:24:41.939979+0000","last_fullsized":"2023-12-14T18:24:41.939979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:09:22.877080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939741+0000","last_change":"2023-12-14T18:24:36.579754+0000","last_active":"2023-12-14T18:24:41.939741+0000","last_peered":"2023-12-14T18:24:41.939741+0000","last_clean":"2023-12-14T18:24:41.939741+0000","last_became_active":"2023-12-14T18:24:36.579337+0000","last_became_peered":"2023-12-14T18:24:36.579337+0000","last_unstale":"2023-12-14T18:24:41.939741+0000","last_undegraded":"2023-12-14T18:24:41.939741+0000","last_fullsized":"2023-12-14T18:24:41.939741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:48:45.306297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212533,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6621,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":117,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":54766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547214,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6016,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739136,"statfs":{"total":95995035648,"available":95988875264,"internally_reserved":0,"allocated":909312,"data_stored":453962,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":10,"up_from":67,"seq":287762808850,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6288,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738864,"statfs":{"total":95995035648,"available":95988596736,"internally_reserved":0,"allocated":991232,"data_stored":1048748,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.6690000000000005,"5min":8.6690000000000005,"15min":8.6690000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":9.5459999999999994,"5min":9.5459999999999994,"15min":9.5459999999999994},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":9.4380000000000006,"5min":9.4380000000000006,"15min":9.4380000000000006},"last":0.504},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.9470000000000001,"5min":5.9470000000000001,"15min":5.9470000000000001},"last":0.443},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":6.1470000000000002,"5min":6.1470000000000002,"15min":6.1470000000000002},"last":0.39500000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":9.4830000000000005,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":0.629},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.64600000000000002}]},{"osd":9,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":9,"up_from":62,"seq":266287972373,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6232,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738920,"statfs":{"total":95995035648,"available":95988654080,"internally_reserved":0,"allocated":933888,"data_stored":458876,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.433}]},{"osd":2,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60699999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.69999999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.66900000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.71499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]}]},{"osd":8,"up_from":55,"seq":236223201305,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":917504,"data_stored":454257,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.625}]},{"osd":1,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.77400000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":1.042}]},{"osd":7,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.872}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":1.016}]},{"osd":10,"last update":"Thu Dec 14 18:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.92400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":7,"up_from":48,"seq":206158430236,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6464,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738688,"statfs":{"total":95995035648,"available":95988416512,"internally_reserved":0,"allocated":974848,"data_stored":1050181,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.434}]},{"osd":1,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.53600000000000003,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.7830000000000004,"5min":5.7830000000000004,"15min":5.7830000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.44700000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.63},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.52000000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77500000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.54400000000000004}]},{"osd":8,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.71199999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.75600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593758,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6272,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738880,"statfs":{"total":95995035648,"available":95988613120,"internally_reserved":0,"allocated":909312,"data_stored":453962,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38500000000000001},"min":{"1min":0.23400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.253,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.37}]},{"osd":1,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70399999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.629}]},{"osd":2,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.745,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.497}]},{"osd":3,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70899999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.46400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.624,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.68600000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:24:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54400000000000004}]},{"osd":9,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47199999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.72899999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7180,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737972,"statfs":{"total":95995035648,"available":95987683328,"internally_reserved":0,"allocated":921600,"data_stored":457755,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47099999999999997,"15min":0.45800000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.47599999999999998,"15min":0.45800000000000002},"min":{"1min":0.32100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.65900000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.498,"15min":0.5},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47799999999999998,"15min":0.46500000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.74199999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.41099999999999998,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.40799999999999997,"15min":0.39800000000000002},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83299999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73099999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.753},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.80500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.77800000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.86899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7172,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737980,"statfs":{"total":95995035648,"available":95987691520,"internally_reserved":0,"allocated":913408,"data_stored":454052,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.60399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.36499999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.51900000000000002,"15min":0.49299999999999999},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.81799999999999995}]},{"osd":3,"last update":"Thu Dec 14 18:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.27300000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35499999999999998},"min":{"1min":0.26700000000000002,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.39000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.502,"15min":0.498},"min":{"1min":0.36899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.71799999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.48899999999999999,"15min":0.47799999999999998},"min":{"1min":0.36399999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.88800000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.93500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.83299999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":1.0569999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7368,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737784,"statfs":{"total":95995035648,"available":95987490816,"internally_reserved":0,"allocated":983040,"data_stored":494168,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.434},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.64200000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.44900000000000001},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.499,"15min":0.47099999999999997},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.51500000000000001,"15min":0.48599999999999999},"min":{"1min":0.39500000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.85599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.46600000000000003},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76100000000000001,"5min":1.179,"15min":1.179},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.39700000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.437,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Dec 14 18:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67600000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36699999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.71899999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.623}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55500000000000005}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":3,"up_from":24,"seq":103079215146,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":925696,"data_stored":454246,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.32800000000000001},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.248,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.497,"15min":0.47499999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Dec 14 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.48399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76500000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73299999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.49199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.497}]},{"osd":6,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.497,"15min":0.499},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":0.755},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36799999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:24:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.628},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48399999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.316},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.35799999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.627}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986215,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6404,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738748,"statfs":{"total":95995035648,"available":95988477952,"internally_reserved":0,"allocated":913408,"data_stored":453984,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48799999999999999,"15min":0.48099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.27300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.45800000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67100000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.45200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.498}]},{"osd":7,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.46600000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:24:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63600000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.59799999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.622}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822691,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6604,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738548,"statfs":{"total":95995035648,"available":95988273152,"internally_reserved":0,"allocated":987136,"data_stored":1045758,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79300000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.433},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.47199999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.77500000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.45500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41399999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.39900000000000002},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.47699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73999999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.997,"15min":0.997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.72399999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66300000000000003}]},{"osd":9,"last update":"Thu Dec 14 18:24:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.629},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64000000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":40206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:25:04.415 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-14T18:25:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:04 smithi160 ceph-mon[43840]: from='client.14883 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:04 smithi160 ceph-mon[43840]: from='client.14889 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:04 smithi160 ceph-mon[43840]: from='client.14895 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:25:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:04 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1612166527' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:04.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.564+0000 7f5f8868c700 1 -- 172.21.15.93:0/461509706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f800ff110 msgr2=0x7f5f800ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 --2- 172.21.15.93:0/461509706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f800ff110 0x7f5f800ff520 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c009990 tx=0x7f5f7c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 -- 172.21.15.93:0/461509706 shutdown_connections 2023-12-14T18:25:04.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 --2- 172.21.15.93:0/461509706 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f800ff110 0x7f5f800ff520 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 --2- 172.21.15.93:0/461509706 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f80100c90 0x7f5f80107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 --2- 172.21.15.93:0/461509706 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f80100300 0x7f5f80100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 -- 172.21.15.93:0/461509706 >> 172.21.15.93:0/461509706 conn(0x7f5f800fa6f0 msgr2=0x7f5f800fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:04.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 -- 172.21.15.93:0/461509706 shutdown_connections 2023-12-14T18:25:04.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 -- 172.21.15.93:0/461509706 wait complete. 2023-12-14T18:25:04.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 Processor -- start 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.565+0000 7f5f8868c700 1 -- start start 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f8868c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800ff110 0x7f5f80194ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f8868c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f80100300 0x7f5f80195410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f8868c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f80100c90 0x7f5f8019c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f8868c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8019ca50 con 0x7f5f80100300 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f8868c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8019cb90 con 0x7f5f800ff110 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f8868c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f8019cd20 con 0x7f5f80100c90 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f85c27700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f80100300 0x7f5f80195410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f85c27700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f80100300 0x7f5f80195410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56932/0 (socket says 172.21.15.93:56932) 2023-12-14T18:25:04.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f85c27700 1 -- 172.21.15.93:0/2812381698 learned_addr learned my addr 172.21.15.93:0/2812381698 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f86428700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800ff110 0x7f5f80194ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f85c27700 1 -- 172.21.15.93:0/2812381698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f80100c90 msgr2=0x7f5f8019c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f86c29700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f80100c90 0x7f5f8019c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f85c27700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f80100c90 0x7f5f8019c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f85c27700 1 -- 172.21.15.93:0/2812381698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800ff110 msgr2=0x7f5f80194ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f85c27700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800ff110 0x7f5f80194ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f85c27700 1 -- 172.21.15.93:0/2812381698 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f7c0095f0 con 0x7f5f80100300 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f86428700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800ff110 0x7f5f80194ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f85c27700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f80100300 0x7f5f80195410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000d980 tx=0x7f5f7000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f777fe700 1 -- 172.21.15.93:0/2812381698 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f7000bb60 con 0x7f5f80100300 2023-12-14T18:25:04.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f777fe700 1 -- 172.21.15.93:0/2812381698 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5f70007640 con 0x7f5f80100300 2023-12-14T18:25:04.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f8010aa70 con 0x7f5f80100300 2023-12-14T18:25:04.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f777fe700 1 -- 172.21.15.93:0/2812381698 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f70004e90 con 0x7f5f80100300 2023-12-14T18:25:04.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.566+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f8010a8e0 con 0x7f5f80100300 2023-12-14T18:25:04.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.567+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f8019d3a0 con 0x7f5f80100300 2023-12-14T18:25:04.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.568+0000 7f5f777fe700 1 -- 172.21.15.93:0/2812381698 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5f7000bcc0 con 0x7f5f80100300 2023-12-14T18:25:04.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.569+0000 7f5f777fe700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5f6c0a1260 0x7f5f6c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:04.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.569+0000 7f5f86428700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5f6c0a1260 0x7f5f6c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:04.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.569+0000 7f5f777fe700 1 -- 172.21.15.93:0/2812381698 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f5f700ccfe0 con 0x7f5f80100300 2023-12-14T18:25:04.573 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.570+0000 7f5f86428700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5f6c0a1260 0x7f5f6c0a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c006010 tx=0x7f5f7c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:04.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.572+0000 7f5f777fe700 1 -- 172.21.15.93:0/2812381698 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5f700963e0 con 0x7f5f80100300 2023-12-14T18:25:04.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.761+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f5f80060f30 con 0x7f5f6c0a1260 2023-12-14T18:25:04.773 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.772+0000 7f5f777fe700 1 -- 172.21.15.93:0/2812381698 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+418218 (secure 0 0 0) 0x7f5f80060f30 con 0x7f5f6c0a1260 2023-12-14T18:25:04.779 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:04.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.776+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5f6c0a1260 msgr2=0x7f5f6c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.776+0000 7f5f8868c700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5f6c0a1260 0x7f5f6c0a3710 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f5f7c006010 tx=0x7f5f7c00b540 comp rx=0 tx=0).stop 2023-12-14T18:25:04.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.776+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f80100300 msgr2=0x7f5f80195410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:04.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.776+0000 7f5f8868c700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f80100300 0x7f5f80195410 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f5f7000d980 tx=0x7f5f7000dd40 comp rx=0 tx=0).stop 2023-12-14T18:25:04.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.776+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 shutdown_connections 2023-12-14T18:25:04.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.777+0000 7f5f8868c700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5f6c0a1260 0x7f5f6c0a3710 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.777+0000 7f5f8868c700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5f80100c90 0x7f5f8019c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.777+0000 7f5f8868c700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5f800ff110 0x7f5f80194ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.777+0000 7f5f8868c700 1 --2- 172.21.15.93:0/2812381698 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5f80100300 0x7f5f80195410 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:04.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.777+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 >> 172.21.15.93:0/2812381698 conn(0x7f5f800fa6f0 msgr2=0x7f5f800fc9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:04.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.777+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 shutdown_connections 2023-12-14T18:25:04.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:04.777+0000 7f5f8868c700 1 -- 172.21.15.93:0/2812381698 wait complete. 2023-12-14T18:25:04.785 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-12-14T18:25:04.790 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":238,"stamp":"2023-12-14T18:25:03.951486+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":597112,"num_objects":55,"num_object_clones":0,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":150,"num_read_kb":133,"num_write":299,"num_write_kb":2971,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":238,"ondisk_log_size":238,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":78888,"kb_used_data":11016,"kb_used_omap":0,"kb_used_meta":67776,"kb_avail":1124862936,"statfs":{"total":1151940427776,"available":1151859646464,"internally_reserved":0,"allocated":11280384,"data_stored":7279949,"data_compressed":246693,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":0,"internal_metadata":69402624},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2294,"num_objects":31,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":71,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":22,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003432"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939919+0000","last_change":"2023-12-14T18:24:36.587133+0000","last_active":"2023-12-14T18:24:41.939919+0000","last_peered":"2023-12-14T18:24:41.939919+0000","last_clean":"2023-12-14T18:24:41.939919+0000","last_became_active":"2023-12-14T18:24:36.586222+0000","last_became_peered":"2023-12-14T18:24:36.586222+0000","last_unstale":"2023-12-14T18:24:41.939919+0000","last_undegraded":"2023-12-14T18:24:41.939919+0000","last_fullsized":"2023-12-14T18:24:41.939919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:51:38.922525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939529+0000","last_change":"2023-12-14T18:24:36.589027+0000","last_active":"2023-12-14T18:24:41.939529+0000","last_peered":"2023-12-14T18:24:41.939529+0000","last_clean":"2023-12-14T18:24:41.939529+0000","last_became_active":"2023-12-14T18:24:36.588785+0000","last_became_peered":"2023-12-14T18:24:36.588785+0000","last_unstale":"2023-12-14T18:24:41.939529+0000","last_undegraded":"2023-12-14T18:24:41.939529+0000","last_fullsized":"2023-12-14T18:24:41.939529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:42.987957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939327+0000","last_change":"2023-12-14T18:24:36.584015+0000","last_active":"2023-12-14T18:24:41.939327+0000","last_peered":"2023-12-14T18:24:41.939327+0000","last_clean":"2023-12-14T18:24:41.939327+0000","last_became_active":"2023-12-14T18:24:36.583735+0000","last_became_peered":"2023-12-14T18:24:36.583735+0000","last_unstale":"2023-12-14T18:24:41.939327+0000","last_undegraded":"2023-12-14T18:24:41.939327+0000","last_fullsized":"2023-12-14T18:24:41.939327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:20:02.823540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940899+0000","last_change":"2023-12-14T18:24:36.584553+0000","last_active":"2023-12-14T18:24:41.940899+0000","last_peered":"2023-12-14T18:24:41.940899+0000","last_clean":"2023-12-14T18:24:41.940899+0000","last_became_active":"2023-12-14T18:24:36.583827+0000","last_became_peered":"2023-12-14T18:24:36.583827+0000","last_unstale":"2023-12-14T18:24:41.940899+0000","last_undegraded":"2023-12-14T18:24:41.940899+0000","last_fullsized":"2023-12-14T18:24:41.940899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:32.928164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095366+0000","last_change":"2023-12-14T18:24:36.578128+0000","last_active":"2023-12-14T18:24:41.095366+0000","last_peered":"2023-12-14T18:24:41.095366+0000","last_clean":"2023-12-14T18:24:41.095366+0000","last_became_active":"2023-12-14T18:24:36.577784+0000","last_became_peered":"2023-12-14T18:24:36.577784+0000","last_unstale":"2023-12-14T18:24:41.095366+0000","last_undegraded":"2023-12-14T18:24:41.095366+0000","last_fullsized":"2023-12-14T18:24:41.095366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:17.890524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940347+0000","last_change":"2023-12-14T18:24:36.588666+0000","last_active":"2023-12-14T18:24:41.940347+0000","last_peered":"2023-12-14T18:24:41.940347+0000","last_clean":"2023-12-14T18:24:41.940347+0000","last_became_active":"2023-12-14T18:24:36.588291+0000","last_became_peered":"2023-12-14T18:24:36.588291+0000","last_unstale":"2023-12-14T18:24:41.940347+0000","last_undegraded":"2023-12-14T18:24:41.940347+0000","last_fullsized":"2023-12-14T18:24:41.940347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:10:59.591257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941626+0000","last_change":"2023-12-14T18:24:36.586637+0000","last_active":"2023-12-14T18:24:41.941626+0000","last_peered":"2023-12-14T18:24:41.941626+0000","last_clean":"2023-12-14T18:24:41.941626+0000","last_became_active":"2023-12-14T18:24:36.583055+0000","last_became_peered":"2023-12-14T18:24:36.583055+0000","last_unstale":"2023-12-14T18:24:41.941626+0000","last_undegraded":"2023-12-14T18:24:41.941626+0000","last_fullsized":"2023-12-14T18:24:41.941626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:31.010603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998916+0000","last_change":"2023-12-14T18:24:36.580206+0000","last_active":"2023-12-14T18:24:40.998916+0000","last_peered":"2023-12-14T18:24:40.998916+0000","last_clean":"2023-12-14T18:24:40.998916+0000","last_became_active":"2023-12-14T18:24:36.579904+0000","last_became_peered":"2023-12-14T18:24:36.579904+0000","last_unstale":"2023-12-14T18:24:40.998916+0000","last_undegraded":"2023-12-14T18:24:40.998916+0000","last_fullsized":"2023-12-14T18:24:40.998916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:15.182919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000918+0000","last_change":"2023-12-14T18:24:36.589890+0000","last_active":"2023-12-14T18:24:41.000918+0000","last_peered":"2023-12-14T18:24:41.000918+0000","last_clean":"2023-12-14T18:24:41.000918+0000","last_became_active":"2023-12-14T18:24:36.588999+0000","last_became_peered":"2023-12-14T18:24:36.588999+0000","last_unstale":"2023-12-14T18:24:41.000918+0000","last_undegraded":"2023-12-14T18:24:41.000918+0000","last_fullsized":"2023-12-14T18:24:41.000918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:59:07.801717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941211+0000","last_change":"2023-12-14T18:24:36.589565+0000","last_active":"2023-12-14T18:24:41.941211+0000","last_peered":"2023-12-14T18:24:41.941211+0000","last_clean":"2023-12-14T18:24:41.941211+0000","last_became_active":"2023-12-14T18:24:36.588822+0000","last_became_peered":"2023-12-14T18:24:36.588822+0000","last_unstale":"2023-12-14T18:24:41.941211+0000","last_undegraded":"2023-12-14T18:24:41.941211+0000","last_fullsized":"2023-12-14T18:24:41.941211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:20:30.545830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941183+0000","last_change":"2023-12-14T18:24:36.593208+0000","last_active":"2023-12-14T18:24:41.941183+0000","last_peered":"2023-12-14T18:24:41.941183+0000","last_clean":"2023-12-14T18:24:41.941183+0000","last_became_active":"2023-12-14T18:24:36.586394+0000","last_became_peered":"2023-12-14T18:24:36.586394+0000","last_unstale":"2023-12-14T18:24:41.941183+0000","last_undegraded":"2023-12-14T18:24:41.941183+0000","last_fullsized":"2023-12-14T18:24:41.941183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:19:50.115165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940763+0000","last_change":"2023-12-14T18:24:36.594254+0000","last_active":"2023-12-14T18:24:41.940763+0000","last_peered":"2023-12-14T18:24:41.940763+0000","last_clean":"2023-12-14T18:24:41.940763+0000","last_became_active":"2023-12-14T18:24:36.593889+0000","last_became_peered":"2023-12-14T18:24:36.593889+0000","last_unstale":"2023-12-14T18:24:41.940763+0000","last_undegraded":"2023-12-14T18:24:41.940763+0000","last_fullsized":"2023-12-14T18:24:41.940763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:50.487280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939911+0000","last_change":"2023-12-14T18:24:36.584971+0000","last_active":"2023-12-14T18:24:41.939911+0000","last_peered":"2023-12-14T18:24:41.939911+0000","last_clean":"2023-12-14T18:24:41.939911+0000","last_became_active":"2023-12-14T18:24:36.584695+0000","last_became_peered":"2023-12-14T18:24:36.584695+0000","last_unstale":"2023-12-14T18:24:41.939911+0000","last_undegraded":"2023-12-14T18:24:41.939911+0000","last_fullsized":"2023-12-14T18:24:41.939911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:43:12.153321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939181+0000","last_change":"2023-12-14T18:24:36.589692+0000","last_active":"2023-12-14T18:24:41.939181+0000","last_peered":"2023-12-14T18:24:41.939181+0000","last_clean":"2023-12-14T18:24:41.939181+0000","last_became_active":"2023-12-14T18:24:36.589415+0000","last_became_peered":"2023-12-14T18:24:36.589415+0000","last_unstale":"2023-12-14T18:24:41.939181+0000","last_undegraded":"2023-12-14T18:24:41.939181+0000","last_fullsized":"2023-12-14T18:24:41.939181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:17:55.715589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.938924+0000","last_change":"2023-12-14T18:24:36.586911+0000","last_active":"2023-12-14T18:24:41.938924+0000","last_peered":"2023-12-14T18:24:41.938924+0000","last_clean":"2023-12-14T18:24:41.938924+0000","last_became_active":"2023-12-14T18:24:36.586600+0000","last_became_peered":"2023-12-14T18:24:36.586600+0000","last_unstale":"2023-12-14T18:24:41.938924+0000","last_undegraded":"2023-12-14T18:24:41.938924+0000","last_fullsized":"2023-12-14T18:24:41.938924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:00:56.150982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000169+0000","last_change":"2023-12-14T18:24:36.584879+0000","last_active":"2023-12-14T18:24:41.000169+0000","last_peered":"2023-12-14T18:24:41.000169+0000","last_clean":"2023-12-14T18:24:41.000169+0000","last_became_active":"2023-12-14T18:24:36.584566+0000","last_became_peered":"2023-12-14T18:24:36.584566+0000","last_unstale":"2023-12-14T18:24:41.000169+0000","last_undegraded":"2023-12-14T18:24:41.000169+0000","last_fullsized":"2023-12-14T18:24:41.000169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:23:00.946127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939646+0000","last_change":"2023-12-14T18:24:36.587420+0000","last_active":"2023-12-14T18:24:41.939646+0000","last_peered":"2023-12-14T18:24:41.939646+0000","last_clean":"2023-12-14T18:24:41.939646+0000","last_became_active":"2023-12-14T18:24:36.586732+0000","last_became_peered":"2023-12-14T18:24:36.586732+0000","last_unstale":"2023-12-14T18:24:41.939646+0000","last_undegraded":"2023-12-14T18:24:41.939646+0000","last_fullsized":"2023-12-14T18:24:41.939646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:06:51.142371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093222+0000","last_change":"2023-12-14T18:24:36.583412+0000","last_active":"2023-12-14T18:24:41.093222+0000","last_peered":"2023-12-14T18:24:41.093222+0000","last_clean":"2023-12-14T18:24:41.093222+0000","last_became_active":"2023-12-14T18:24:36.583022+0000","last_became_peered":"2023-12-14T18:24:36.583022+0000","last_unstale":"2023-12-14T18:24:41.093222+0000","last_undegraded":"2023-12-14T18:24:41.093222+0000","last_fullsized":"2023-12-14T18:24:41.093222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:35:26.281488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001176+0000","last_change":"2023-12-14T18:24:36.575232+0000","last_active":"2023-12-14T18:24:41.001176+0000","last_peered":"2023-12-14T18:24:41.001176+0000","last_clean":"2023-12-14T18:24:41.001176+0000","last_became_active":"2023-12-14T18:24:36.574573+0000","last_became_peered":"2023-12-14T18:24:36.574573+0000","last_unstale":"2023-12-14T18:24:41.001176+0000","last_undegraded":"2023-12-14T18:24:41.001176+0000","last_fullsized":"2023-12-14T18:24:41.001176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:09:44.789329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000573+0000","last_change":"2023-12-14T18:24:40.605077+0000","last_active":"2023-12-14T18:24:41.000573+0000","last_peered":"2023-12-14T18:24:41.000573+0000","last_clean":"2023-12-14T18:24:41.000573+0000","last_became_active":"2023-12-14T18:24:40.604646+0000","last_became_peered":"2023-12-14T18:24:40.604646+0000","last_unstale":"2023-12-14T18:24:41.000573+0000","last_undegraded":"2023-12-14T18:24:41.000573+0000","last_fullsized":"2023-12-14T18:24:41.000573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:46:49.342657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.768860+0000","last_change":"2023-12-14T18:24:36.177449+0000","last_active":"2023-12-14T18:25:00.768860+0000","last_peered":"2023-12-14T18:25:00.768860+0000","last_clean":"2023-12-14T18:25:00.768860+0000","last_became_active":"2023-12-14T18:24:36.177114+0000","last_became_peered":"2023-12-14T18:24:36.177114+0000","last_unstale":"2023-12-14T18:25:00.768860+0000","last_undegraded":"2023-12-14T18:25:00.768860+0000","last_fullsized":"2023-12-14T18:25:00.768860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:30.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999464+0000","last_change":"2023-12-14T18:24:36.574204+0000","last_active":"2023-12-14T18:24:40.999464+0000","last_peered":"2023-12-14T18:24:40.999464+0000","last_clean":"2023-12-14T18:24:40.999464+0000","last_became_active":"2023-12-14T18:24:36.573923+0000","last_became_peered":"2023-12-14T18:24:36.573923+0000","last_unstale":"2023-12-14T18:24:40.999464+0000","last_undegraded":"2023-12-14T18:24:40.999464+0000","last_fullsized":"2023-12-14T18:24:40.999464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:20:32.637706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939617+0000","last_change":"2023-12-14T18:24:40.600254+0000","last_active":"2023-12-14T18:24:41.939617+0000","last_peered":"2023-12-14T18:24:41.939617+0000","last_clean":"2023-12-14T18:24:41.939617+0000","last_became_active":"2023-12-14T18:24:40.599894+0000","last_became_peered":"2023-12-14T18:24:40.599894+0000","last_unstale":"2023-12-14T18:24:41.939617+0000","last_undegraded":"2023-12-14T18:24:41.939617+0000","last_fullsized":"2023-12-14T18:24:41.939617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:00.116137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999521+0000","last_change":"2023-12-14T18:24:36.170109+0000","last_active":"2023-12-14T18:24:40.999521+0000","last_peered":"2023-12-14T18:24:40.999521+0000","last_clean":"2023-12-14T18:24:40.999521+0000","last_became_active":"2023-12-14T18:24:36.169629+0000","last_became_peered":"2023-12-14T18:24:36.169629+0000","last_unstale":"2023-12-14T18:24:40.999521+0000","last_undegraded":"2023-12-14T18:24:40.999521+0000","last_fullsized":"2023-12-14T18:24:40.999521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:41:16.457493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939537+0000","last_change":"2023-12-14T18:24:36.582757+0000","last_active":"2023-12-14T18:24:41.939537+0000","last_peered":"2023-12-14T18:24:41.939537+0000","last_clean":"2023-12-14T18:24:41.939537+0000","last_became_active":"2023-12-14T18:24:36.582457+0000","last_became_peered":"2023-12-14T18:24:36.582457+0000","last_unstale":"2023-12-14T18:24:41.939537+0000","last_undegraded":"2023-12-14T18:24:41.939537+0000","last_fullsized":"2023-12-14T18:24:41.939537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:29:05.124428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940384+0000","last_change":"2023-12-14T18:24:40.599695+0000","last_active":"2023-12-14T18:24:41.940384+0000","last_peered":"2023-12-14T18:24:41.940384+0000","last_clean":"2023-12-14T18:24:41.940384+0000","last_became_active":"2023-12-14T18:24:40.599239+0000","last_became_peered":"2023-12-14T18:24:40.599239+0000","last_unstale":"2023-12-14T18:24:41.940384+0000","last_undegraded":"2023-12-14T18:24:41.940384+0000","last_fullsized":"2023-12-14T18:24:41.940384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:40.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095870+0000","last_change":"2023-12-14T18:24:36.177946+0000","last_active":"2023-12-14T18:24:41.095870+0000","last_peered":"2023-12-14T18:24:41.095870+0000","last_clean":"2023-12-14T18:24:41.095870+0000","last_became_active":"2023-12-14T18:24:36.177480+0000","last_became_peered":"2023-12-14T18:24:36.177480+0000","last_unstale":"2023-12-14T18:24:41.095870+0000","last_undegraded":"2023-12-14T18:24:41.095870+0000","last_fullsized":"2023-12-14T18:24:41.095870+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:42:02.447975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000436+0000","last_change":"2023-12-14T18:24:36.587951+0000","last_active":"2023-12-14T18:24:41.000436+0000","last_peered":"2023-12-14T18:24:41.000436+0000","last_clean":"2023-12-14T18:24:41.000436+0000","last_became_active":"2023-12-14T18:24:36.587539+0000","last_became_peered":"2023-12-14T18:24:36.587539+0000","last_unstale":"2023-12-14T18:24:41.000436+0000","last_undegraded":"2023-12-14T18:24:41.000436+0000","last_fullsized":"2023-12-14T18:24:41.000436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:53:03.285569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941747+0000","last_change":"2023-12-14T18:24:40.600008+0000","last_active":"2023-12-14T18:24:41.941747+0000","last_peered":"2023-12-14T18:24:41.941747+0000","last_clean":"2023-12-14T18:24:41.941747+0000","last_became_active":"2023-12-14T18:24:40.599596+0000","last_became_peered":"2023-12-14T18:24:40.599596+0000","last_unstale":"2023-12-14T18:24:41.941747+0000","last_undegraded":"2023-12-14T18:24:41.941747+0000","last_fullsized":"2023-12-14T18:24:41.941747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:10:10.770024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095629+0000","last_change":"2023-12-14T18:24:36.176569+0000","last_active":"2023-12-14T18:24:41.095629+0000","last_peered":"2023-12-14T18:24:41.095629+0000","last_clean":"2023-12-14T18:24:41.095629+0000","last_became_active":"2023-12-14T18:24:36.176043+0000","last_became_peered":"2023-12-14T18:24:36.176043+0000","last_unstale":"2023-12-14T18:24:41.095629+0000","last_undegraded":"2023-12-14T18:24:41.095629+0000","last_fullsized":"2023-12-14T18:24:41.095629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:10:44.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149101+0000","last_change":"2023-12-14T18:24:36.593761+0000","last_active":"2023-12-14T18:24:41.149101+0000","last_peered":"2023-12-14T18:24:41.149101+0000","last_clean":"2023-12-14T18:24:41.149101+0000","last_became_active":"2023-12-14T18:24:36.593452+0000","last_became_peered":"2023-12-14T18:24:36.593452+0000","last_unstale":"2023-12-14T18:24:41.149101+0000","last_undegraded":"2023-12-14T18:24:41.149101+0000","last_fullsized":"2023-12-14T18:24:41.149101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:56:34.794908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001069+0000","last_change":"2023-12-14T18:24:40.598064+0000","last_active":"2023-12-14T18:24:41.001069+0000","last_peered":"2023-12-14T18:24:41.001069+0000","last_clean":"2023-12-14T18:24:41.001069+0000","last_became_active":"2023-12-14T18:24:40.597672+0000","last_became_peered":"2023-12-14T18:24:40.597672+0000","last_unstale":"2023-12-14T18:24:41.001069+0000","last_undegraded":"2023-12-14T18:24:41.001069+0000","last_fullsized":"2023-12-14T18:24:41.001069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:41:54.138227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'5","reported_seq":31,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.760893+0000","last_change":"2023-12-14T18:24:36.176288+0000","last_active":"2023-12-14T18:25:00.760893+0000","last_peered":"2023-12-14T18:25:00.760893+0000","last_clean":"2023-12-14T18:25:00.760893+0000","last_became_active":"2023-12-14T18:24:36.175965+0000","last_became_peered":"2023-12-14T18:24:36.175965+0000","last_unstale":"2023-12-14T18:25:00.760893+0000","last_undegraded":"2023-12-14T18:25:00.760893+0000","last_fullsized":"2023-12-14T18:25:00.760893+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:00:50.888829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":574,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095492+0000","last_change":"2023-12-14T18:24:36.583687+0000","last_active":"2023-12-14T18:24:41.095492+0000","last_peered":"2023-12-14T18:24:41.095492+0000","last_clean":"2023-12-14T18:24:41.095492+0000","last_became_active":"2023-12-14T18:24:36.583047+0000","last_became_peered":"2023-12-14T18:24:36.583047+0000","last_unstale":"2023-12-14T18:24:41.095492+0000","last_undegraded":"2023-12-14T18:24:41.095492+0000","last_fullsized":"2023-12-14T18:24:41.095492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:26:10.095059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001102+0000","last_change":"2023-12-14T18:24:40.605217+0000","last_active":"2023-12-14T18:24:41.001102+0000","last_peered":"2023-12-14T18:24:41.001102+0000","last_clean":"2023-12-14T18:24:41.001102+0000","last_became_active":"2023-12-14T18:24:40.604807+0000","last_became_peered":"2023-12-14T18:24:40.604807+0000","last_unstale":"2023-12-14T18:24:41.001102+0000","last_undegraded":"2023-12-14T18:24:41.001102+0000","last_fullsized":"2023-12-14T18:24:41.001102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:48:27.699822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767627+0000","last_change":"2023-12-14T18:24:36.174875+0000","last_active":"2023-12-14T18:25:00.767627+0000","last_peered":"2023-12-14T18:25:00.767627+0000","last_clean":"2023-12-14T18:25:00.767627+0000","last_became_active":"2023-12-14T18:24:36.173984+0000","last_became_peered":"2023-12-14T18:24:36.173984+0000","last_unstale":"2023-12-14T18:25:00.767627+0000","last_undegraded":"2023-12-14T18:25:00.767627+0000","last_fullsized":"2023-12-14T18:25:00.767627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:24:27.753878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149194+0000","last_change":"2023-12-14T18:24:36.580563+0000","last_active":"2023-12-14T18:24:41.149194+0000","last_peered":"2023-12-14T18:24:41.149194+0000","last_clean":"2023-12-14T18:24:41.149194+0000","last_became_active":"2023-12-14T18:24:36.580196+0000","last_became_peered":"2023-12-14T18:24:36.580196+0000","last_unstale":"2023-12-14T18:24:41.149194+0000","last_undegraded":"2023-12-14T18:24:41.149194+0000","last_fullsized":"2023-12-14T18:24:41.149194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:44:38.053372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939735+0000","last_change":"2023-12-14T18:24:40.601223+0000","last_active":"2023-12-14T18:24:41.939735+0000","last_peered":"2023-12-14T18:24:41.939735+0000","last_clean":"2023-12-14T18:24:41.939735+0000","last_became_active":"2023-12-14T18:24:40.600795+0000","last_became_peered":"2023-12-14T18:24:40.600795+0000","last_unstale":"2023-12-14T18:24:41.939735+0000","last_undegraded":"2023-12-14T18:24:41.939735+0000","last_fullsized":"2023-12-14T18:24:41.939735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:27:49.251012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767406+0000","last_change":"2023-12-14T18:24:36.173600+0000","last_active":"2023-12-14T18:25:00.767406+0000","last_peered":"2023-12-14T18:25:00.767406+0000","last_clean":"2023-12-14T18:25:00.767406+0000","last_became_active":"2023-12-14T18:24:36.173272+0000","last_became_peered":"2023-12-14T18:24:36.173272+0000","last_unstale":"2023-12-14T18:25:00.767406+0000","last_undegraded":"2023-12-14T18:25:00.767406+0000","last_fullsized":"2023-12-14T18:25:00.767406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:23.589714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998814+0000","last_change":"2023-12-14T18:24:36.575060+0000","last_active":"2023-12-14T18:24:40.998814+0000","last_peered":"2023-12-14T18:24:40.998814+0000","last_clean":"2023-12-14T18:24:40.998814+0000","last_became_active":"2023-12-14T18:24:36.574749+0000","last_became_peered":"2023-12-14T18:24:36.574749+0000","last_unstale":"2023-12-14T18:24:40.998814+0000","last_undegraded":"2023-12-14T18:24:40.998814+0000","last_fullsized":"2023-12-14T18:24:40.998814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:44:44.163762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939489+0000","last_change":"2023-12-14T18:24:40.605431+0000","last_active":"2023-12-14T18:24:41.939489+0000","last_peered":"2023-12-14T18:24:41.939489+0000","last_clean":"2023-12-14T18:24:41.939489+0000","last_became_active":"2023-12-14T18:24:40.604951+0000","last_became_peered":"2023-12-14T18:24:40.604951+0000","last_unstale":"2023-12-14T18:24:41.939489+0000","last_undegraded":"2023-12-14T18:24:41.939489+0000","last_fullsized":"2023-12-14T18:24:41.939489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:46:36.552203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148653+0000","last_change":"2023-12-14T18:24:36.175493+0000","last_active":"2023-12-14T18:24:41.148653+0000","last_peered":"2023-12-14T18:24:41.148653+0000","last_clean":"2023-12-14T18:24:41.148653+0000","last_became_active":"2023-12-14T18:24:36.175205+0000","last_became_peered":"2023-12-14T18:24:36.175205+0000","last_unstale":"2023-12-14T18:24:41.148653+0000","last_undegraded":"2023-12-14T18:24:41.148653+0000","last_fullsized":"2023-12-14T18:24:41.148653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:52:40.611249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092822+0000","last_change":"2023-12-14T18:24:36.587218+0000","last_active":"2023-12-14T18:24:41.092822+0000","last_peered":"2023-12-14T18:24:41.092822+0000","last_clean":"2023-12-14T18:24:41.092822+0000","last_became_active":"2023-12-14T18:24:36.586910+0000","last_became_peered":"2023-12-14T18:24:36.586910+0000","last_unstale":"2023-12-14T18:24:41.092822+0000","last_undegraded":"2023-12-14T18:24:41.092822+0000","last_fullsized":"2023-12-14T18:24:41.092822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:04.691462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999151+0000","last_change":"2023-12-14T18:24:40.598409+0000","last_active":"2023-12-14T18:24:40.999151+0000","last_peered":"2023-12-14T18:24:40.999151+0000","last_clean":"2023-12-14T18:24:40.999151+0000","last_became_active":"2023-12-14T18:24:40.598011+0000","last_became_peered":"2023-12-14T18:24:40.598011+0000","last_unstale":"2023-12-14T18:24:40.999151+0000","last_undegraded":"2023-12-14T18:24:40.999151+0000","last_fullsized":"2023-12-14T18:24:40.999151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:30.946044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039152+0000","last_change":"2023-12-14T18:24:36.172277+0000","last_active":"2023-12-14T18:25:00.039152+0000","last_peered":"2023-12-14T18:25:00.039152+0000","last_clean":"2023-12-14T18:25:00.039152+0000","last_became_active":"2023-12-14T18:24:36.171907+0000","last_became_peered":"2023-12-14T18:24:36.171907+0000","last_unstale":"2023-12-14T18:25:00.039152+0000","last_undegraded":"2023-12-14T18:25:00.039152+0000","last_fullsized":"2023-12-14T18:25:00.039152+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:45:12.002124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000922+0000","last_change":"2023-12-14T18:24:36.584783+0000","last_active":"2023-12-14T18:24:41.000922+0000","last_peered":"2023-12-14T18:24:41.000922+0000","last_clean":"2023-12-14T18:24:41.000922+0000","last_became_active":"2023-12-14T18:24:36.584424+0000","last_became_peered":"2023-12-14T18:24:36.584424+0000","last_unstale":"2023-12-14T18:24:41.000922+0000","last_undegraded":"2023-12-14T18:24:41.000922+0000","last_fullsized":"2023-12-14T18:24:41.000922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:13:46.074921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092851+0000","last_change":"2023-12-14T18:24:40.603176+0000","last_active":"2023-12-14T18:24:41.092851+0000","last_peered":"2023-12-14T18:24:41.092851+0000","last_clean":"2023-12-14T18:24:41.092851+0000","last_became_active":"2023-12-14T18:24:40.602868+0000","last_became_peered":"2023-12-14T18:24:40.602868+0000","last_unstale":"2023-12-14T18:24:41.092851+0000","last_undegraded":"2023-12-14T18:24:41.092851+0000","last_fullsized":"2023-12-14T18:24:41.092851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:02:23.455860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.037799+0000","last_change":"2023-12-14T18:24:36.177638+0000","last_active":"2023-12-14T18:25:00.037799+0000","last_peered":"2023-12-14T18:25:00.037799+0000","last_clean":"2023-12-14T18:25:00.037799+0000","last_became_active":"2023-12-14T18:24:36.177335+0000","last_became_peered":"2023-12-14T18:24:36.177335+0000","last_unstale":"2023-12-14T18:25:00.037799+0000","last_undegraded":"2023-12-14T18:25:00.037799+0000","last_fullsized":"2023-12-14T18:25:00.037799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:50:02.813317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939828+0000","last_change":"2023-12-14T18:24:36.577508+0000","last_active":"2023-12-14T18:24:41.939828+0000","last_peered":"2023-12-14T18:24:41.939828+0000","last_clean":"2023-12-14T18:24:41.939828+0000","last_became_active":"2023-12-14T18:24:36.577196+0000","last_became_peered":"2023-12-14T18:24:36.577196+0000","last_unstale":"2023-12-14T18:24:41.939828+0000","last_undegraded":"2023-12-14T18:24:41.939828+0000","last_fullsized":"2023-12-14T18:24:41.939828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:32:16.278415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941240+0000","last_change":"2023-12-14T18:24:40.594816+0000","last_active":"2023-12-14T18:24:41.941240+0000","last_peered":"2023-12-14T18:24:41.941240+0000","last_clean":"2023-12-14T18:24:41.941240+0000","last_became_active":"2023-12-14T18:24:40.594371+0000","last_became_peered":"2023-12-14T18:24:40.594371+0000","last_unstale":"2023-12-14T18:24:41.941240+0000","last_undegraded":"2023-12-14T18:24:41.941240+0000","last_fullsized":"2023-12-14T18:24:41.941240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:40:46.343094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039258+0000","last_change":"2023-12-14T18:24:36.174238+0000","last_active":"2023-12-14T18:25:00.039258+0000","last_peered":"2023-12-14T18:25:00.039258+0000","last_clean":"2023-12-14T18:25:00.039258+0000","last_became_active":"2023-12-14T18:24:36.173373+0000","last_became_peered":"2023-12-14T18:24:36.173373+0000","last_unstale":"2023-12-14T18:25:00.039258+0000","last_undegraded":"2023-12-14T18:25:00.039258+0000","last_fullsized":"2023-12-14T18:25:00.039258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:40:50.640090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000757+0000","last_change":"2023-12-14T18:24:36.588589+0000","last_active":"2023-12-14T18:24:41.000757+0000","last_peered":"2023-12-14T18:24:41.000757+0000","last_clean":"2023-12-14T18:24:41.000757+0000","last_became_active":"2023-12-14T18:24:36.588269+0000","last_became_peered":"2023-12-14T18:24:36.588269+0000","last_unstale":"2023-12-14T18:24:41.000757+0000","last_undegraded":"2023-12-14T18:24:41.000757+0000","last_fullsized":"2023-12-14T18:24:41.000757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:12.316505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092841+0000","last_change":"2023-12-14T18:24:40.601333+0000","last_active":"2023-12-14T18:24:41.092841+0000","last_peered":"2023-12-14T18:24:41.092841+0000","last_clean":"2023-12-14T18:24:41.092841+0000","last_became_active":"2023-12-14T18:24:40.600966+0000","last_became_peered":"2023-12-14T18:24:40.600966+0000","last_unstale":"2023-12-14T18:24:41.092841+0000","last_undegraded":"2023-12-14T18:24:41.092841+0000","last_fullsized":"2023-12-14T18:24:41.092841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:36.433276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'11","reported_seq":37,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.759760+0000","last_change":"2023-12-14T18:24:36.177299+0000","last_active":"2023-12-14T18:25:00.759760+0000","last_peered":"2023-12-14T18:25:00.759760+0000","last_clean":"2023-12-14T18:25:00.759760+0000","last_became_active":"2023-12-14T18:24:36.176971+0000","last_became_peered":"2023-12-14T18:24:36.176971+0000","last_unstale":"2023-12-14T18:25:00.759760+0000","last_undegraded":"2023-12-14T18:25:00.759760+0000","last_fullsized":"2023-12-14T18:25:00.759760+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:02:46.083153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3793,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":6,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095663+0000","last_change":"2023-12-14T18:24:36.586315+0000","last_active":"2023-12-14T18:24:41.095663+0000","last_peered":"2023-12-14T18:24:41.095663+0000","last_clean":"2023-12-14T18:24:41.095663+0000","last_became_active":"2023-12-14T18:24:36.586000+0000","last_became_peered":"2023-12-14T18:24:36.586000+0000","last_unstale":"2023-12-14T18:24:41.095663+0000","last_undegraded":"2023-12-14T18:24:41.095663+0000","last_fullsized":"2023-12-14T18:24:41.095663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:19:55.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149539+0000","last_change":"2023-12-14T18:24:40.596973+0000","last_active":"2023-12-14T18:24:41.149539+0000","last_peered":"2023-12-14T18:24:41.149539+0000","last_clean":"2023-12-14T18:24:41.149539+0000","last_became_active":"2023-12-14T18:24:40.596355+0000","last_became_peered":"2023-12-14T18:24:40.596355+0000","last_unstale":"2023-12-14T18:24:41.149539+0000","last_undegraded":"2023-12-14T18:24:41.149539+0000","last_fullsized":"2023-12-14T18:24:41.149539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:19:02.715446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767785+0000","last_change":"2023-12-14T18:24:36.175287+0000","last_active":"2023-12-14T18:25:00.767785+0000","last_peered":"2023-12-14T18:25:00.767785+0000","last_clean":"2023-12-14T18:25:00.767785+0000","last_became_active":"2023-12-14T18:24:36.174952+0000","last_became_peered":"2023-12-14T18:24:36.174952+0000","last_unstale":"2023-12-14T18:25:00.767785+0000","last_undegraded":"2023-12-14T18:25:00.767785+0000","last_fullsized":"2023-12-14T18:25:00.767785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:44:43.661729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940588+0000","last_change":"2023-12-14T18:24:36.582314+0000","last_active":"2023-12-14T18:24:41.940588+0000","last_peered":"2023-12-14T18:24:41.940588+0000","last_clean":"2023-12-14T18:24:41.940588+0000","last_became_active":"2023-12-14T18:24:36.581963+0000","last_became_peered":"2023-12-14T18:24:36.581963+0000","last_unstale":"2023-12-14T18:24:41.940588+0000","last_undegraded":"2023-12-14T18:24:41.940588+0000","last_fullsized":"2023-12-14T18:24:41.940588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:02:38.477328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939741+0000","last_change":"2023-12-14T18:24:40.599979+0000","last_active":"2023-12-14T18:24:41.939741+0000","last_peered":"2023-12-14T18:24:41.939741+0000","last_clean":"2023-12-14T18:24:41.939741+0000","last_became_active":"2023-12-14T18:24:40.599258+0000","last_became_peered":"2023-12-14T18:24:40.599258+0000","last_unstale":"2023-12-14T18:24:41.939741+0000","last_undegraded":"2023-12-14T18:24:41.939741+0000","last_fullsized":"2023-12-14T18:24:41.939741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:06.783797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039434+0000","last_change":"2023-12-14T18:24:36.175768+0000","last_active":"2023-12-14T18:25:00.039434+0000","last_peered":"2023-12-14T18:25:00.039434+0000","last_clean":"2023-12-14T18:25:00.039434+0000","last_became_active":"2023-12-14T18:24:36.174973+0000","last_became_peered":"2023-12-14T18:24:36.174973+0000","last_unstale":"2023-12-14T18:25:00.039434+0000","last_undegraded":"2023-12-14T18:25:00.039434+0000","last_fullsized":"2023-12-14T18:25:00.039434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:54.191789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940662+0000","last_change":"2023-12-14T18:24:36.591856+0000","last_active":"2023-12-14T18:24:41.940662+0000","last_peered":"2023-12-14T18:24:41.940662+0000","last_clean":"2023-12-14T18:24:41.940662+0000","last_became_active":"2023-12-14T18:24:36.591554+0000","last_became_peered":"2023-12-14T18:24:36.591554+0000","last_unstale":"2023-12-14T18:24:41.940662+0000","last_undegraded":"2023-12-14T18:24:41.940662+0000","last_fullsized":"2023-12-14T18:24:41.940662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:31:39.565049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093250+0000","last_change":"2023-12-14T18:24:40.603794+0000","last_active":"2023-12-14T18:24:41.093250+0000","last_peered":"2023-12-14T18:24:41.093250+0000","last_clean":"2023-12-14T18:24:41.093250+0000","last_became_active":"2023-12-14T18:24:40.603311+0000","last_became_peered":"2023-12-14T18:24:40.603311+0000","last_unstale":"2023-12-14T18:24:41.093250+0000","last_undegraded":"2023-12-14T18:24:41.093250+0000","last_fullsized":"2023-12-14T18:24:41.093250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:11:14.354480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939789+0000","last_change":"2023-12-14T18:24:36.177411+0000","last_active":"2023-12-14T18:24:41.939789+0000","last_peered":"2023-12-14T18:24:41.939789+0000","last_clean":"2023-12-14T18:24:41.939789+0000","last_became_active":"2023-12-14T18:24:36.177047+0000","last_became_peered":"2023-12-14T18:24:36.177047+0000","last_unstale":"2023-12-14T18:24:41.939789+0000","last_undegraded":"2023-12-14T18:24:41.939789+0000","last_fullsized":"2023-12-14T18:24:41.939789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:58:22.871429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939900+0000","last_change":"2023-12-14T18:24:36.597335+0000","last_active":"2023-12-14T18:24:41.939900+0000","last_peered":"2023-12-14T18:24:41.939900+0000","last_clean":"2023-12-14T18:24:41.939900+0000","last_became_active":"2023-12-14T18:24:36.596900+0000","last_became_peered":"2023-12-14T18:24:36.596900+0000","last_unstale":"2023-12-14T18:24:41.939900+0000","last_undegraded":"2023-12-14T18:24:41.939900+0000","last_fullsized":"2023-12-14T18:24:41.939900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:07:51.506541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940190+0000","last_change":"2023-12-14T18:24:40.607042+0000","last_active":"2023-12-14T18:24:41.940190+0000","last_peered":"2023-12-14T18:24:41.940190+0000","last_clean":"2023-12-14T18:24:41.940190+0000","last_became_active":"2023-12-14T18:24:40.606563+0000","last_became_peered":"2023-12-14T18:24:40.606563+0000","last_unstale":"2023-12-14T18:24:41.940190+0000","last_undegraded":"2023-12-14T18:24:41.940190+0000","last_fullsized":"2023-12-14T18:24:41.940190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:31:29.839031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'10","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.038304+0000","last_change":"2023-12-14T18:24:36.171994+0000","last_active":"2023-12-14T18:25:00.038304+0000","last_peered":"2023-12-14T18:25:00.038304+0000","last_clean":"2023-12-14T18:25:00.038304+0000","last_became_active":"2023-12-14T18:24:36.171661+0000","last_became_peered":"2023-12-14T18:24:36.171661+0000","last_unstale":"2023-12-14T18:25:00.038304+0000","last_undegraded":"2023-12-14T18:25:00.038304+0000","last_fullsized":"2023-12-14T18:25:00.038304+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:33:33.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148674+0000","last_change":"2023-12-14T18:24:36.577818+0000","last_active":"2023-12-14T18:24:41.148674+0000","last_peered":"2023-12-14T18:24:41.148674+0000","last_clean":"2023-12-14T18:24:41.148674+0000","last_became_active":"2023-12-14T18:24:36.577356+0000","last_became_peered":"2023-12-14T18:24:36.577356+0000","last_unstale":"2023-12-14T18:24:41.148674+0000","last_undegraded":"2023-12-14T18:24:41.148674+0000","last_fullsized":"2023-12-14T18:24:41.148674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:22:05.663769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148886+0000","last_change":"2023-12-14T18:24:40.601995+0000","last_active":"2023-12-14T18:24:41.148886+0000","last_peered":"2023-12-14T18:24:41.148886+0000","last_clean":"2023-12-14T18:24:41.148886+0000","last_became_active":"2023-12-14T18:24:40.601571+0000","last_became_peered":"2023-12-14T18:24:40.601571+0000","last_unstale":"2023-12-14T18:24:41.148886+0000","last_undegraded":"2023-12-14T18:24:41.148886+0000","last_fullsized":"2023-12-14T18:24:41.148886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:30:58.566301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"79'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095876+0000","last_change":"2023-12-14T18:24:36.168661+0000","last_active":"2023-12-14T18:24:41.095876+0000","last_peered":"2023-12-14T18:24:41.095876+0000","last_clean":"2023-12-14T18:24:41.095876+0000","last_became_active":"2023-12-14T18:24:36.168299+0000","last_became_peered":"2023-12-14T18:24:36.168299+0000","last_unstale":"2023-12-14T18:24:41.095876+0000","last_undegraded":"2023-12-14T18:24:41.095876+0000","last_fullsized":"2023-12-14T18:24:41.095876+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:55.288902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939900+0000","last_change":"2023-12-14T18:24:36.578417+0000","last_active":"2023-12-14T18:24:41.939900+0000","last_peered":"2023-12-14T18:24:41.939900+0000","last_clean":"2023-12-14T18:24:41.939900+0000","last_became_active":"2023-12-14T18:24:36.578080+0000","last_became_peered":"2023-12-14T18:24:36.578080+0000","last_unstale":"2023-12-14T18:24:41.939900+0000","last_undegraded":"2023-12-14T18:24:41.939900+0000","last_fullsized":"2023-12-14T18:24:41.939900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:07:18.978037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095510+0000","last_change":"2023-12-14T18:24:40.771866+0000","last_active":"2023-12-14T18:24:41.095510+0000","last_peered":"2023-12-14T18:24:41.095510+0000","last_clean":"2023-12-14T18:24:41.095510+0000","last_became_active":"2023-12-14T18:24:40.771324+0000","last_became_peered":"2023-12-14T18:24:40.771324+0000","last_unstale":"2023-12-14T18:24:41.095510+0000","last_undegraded":"2023-12-14T18:24:41.095510+0000","last_fullsized":"2023-12-14T18:24:41.095510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:20:19.058538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093047+0000","last_change":"2023-12-14T18:24:36.170489+0000","last_active":"2023-12-14T18:24:41.093047+0000","last_peered":"2023-12-14T18:24:41.093047+0000","last_clean":"2023-12-14T18:24:41.093047+0000","last_became_active":"2023-12-14T18:24:36.170161+0000","last_became_peered":"2023-12-14T18:24:36.170161+0000","last_unstale":"2023-12-14T18:24:41.093047+0000","last_undegraded":"2023-12-14T18:24:41.093047+0000","last_fullsized":"2023-12-14T18:24:41.093047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:33:56.690387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940761+0000","last_change":"2023-12-14T18:24:36.591094+0000","last_active":"2023-12-14T18:24:41.940761+0000","last_peered":"2023-12-14T18:24:41.940761+0000","last_clean":"2023-12-14T18:24:41.940761+0000","last_became_active":"2023-12-14T18:24:36.590779+0000","last_became_peered":"2023-12-14T18:24:36.590779+0000","last_unstale":"2023-12-14T18:24:41.940761+0000","last_undegraded":"2023-12-14T18:24:41.940761+0000","last_fullsized":"2023-12-14T18:24:41.940761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:16:00.885373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940628+0000","last_change":"2023-12-14T18:24:40.604125+0000","last_active":"2023-12-14T18:24:41.940628+0000","last_peered":"2023-12-14T18:24:41.940628+0000","last_clean":"2023-12-14T18:24:41.940628+0000","last_became_active":"2023-12-14T18:24:40.603242+0000","last_became_peered":"2023-12-14T18:24:40.603242+0000","last_unstale":"2023-12-14T18:24:41.940628+0000","last_undegraded":"2023-12-14T18:24:41.940628+0000","last_fullsized":"2023-12-14T18:24:41.940628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:52:35.630493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940851+0000","last_change":"2023-12-14T18:24:36.177567+0000","last_active":"2023-12-14T18:24:41.940851+0000","last_peered":"2023-12-14T18:24:41.940851+0000","last_clean":"2023-12-14T18:24:41.940851+0000","last_became_active":"2023-12-14T18:24:36.176949+0000","last_became_peered":"2023-12-14T18:24:36.176949+0000","last_unstale":"2023-12-14T18:24:41.940851+0000","last_undegraded":"2023-12-14T18:24:41.940851+0000","last_fullsized":"2023-12-14T18:24:41.940851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:13:55.523229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940032+0000","last_change":"2023-12-14T18:24:36.594224+0000","last_active":"2023-12-14T18:24:41.940032+0000","last_peered":"2023-12-14T18:24:41.940032+0000","last_clean":"2023-12-14T18:24:41.940032+0000","last_became_active":"2023-12-14T18:24:36.593859+0000","last_became_peered":"2023-12-14T18:24:36.593859+0000","last_unstale":"2023-12-14T18:24:41.940032+0000","last_undegraded":"2023-12-14T18:24:41.940032+0000","last_fullsized":"2023-12-14T18:24:41.940032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:59:04.044943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940173+0000","last_change":"2023-12-14T18:24:40.771814+0000","last_active":"2023-12-14T18:24:41.940173+0000","last_peered":"2023-12-14T18:24:41.940173+0000","last_clean":"2023-12-14T18:24:41.940173+0000","last_became_active":"2023-12-14T18:24:40.771387+0000","last_became_peered":"2023-12-14T18:24:40.771387+0000","last_unstale":"2023-12-14T18:24:41.940173+0000","last_undegraded":"2023-12-14T18:24:41.940173+0000","last_fullsized":"2023-12-14T18:24:41.940173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:22.744059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148815+0000","last_change":"2023-12-14T18:24:36.588826+0000","last_active":"2023-12-14T18:24:41.148815+0000","last_peered":"2023-12-14T18:24:41.148815+0000","last_clean":"2023-12-14T18:24:41.148815+0000","last_became_active":"2023-12-14T18:24:36.588320+0000","last_became_peered":"2023-12-14T18:24:36.588320+0000","last_unstale":"2023-12-14T18:24:41.148815+0000","last_undegraded":"2023-12-14T18:24:41.148815+0000","last_fullsized":"2023-12-14T18:24:41.148815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:15:09.573489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.768534+0000","last_change":"2023-12-14T18:24:36.173995+0000","last_active":"2023-12-14T18:25:00.768534+0000","last_peered":"2023-12-14T18:25:00.768534+0000","last_clean":"2023-12-14T18:25:00.768534+0000","last_became_active":"2023-12-14T18:24:36.173381+0000","last_became_peered":"2023-12-14T18:24:36.173381+0000","last_unstale":"2023-12-14T18:25:00.768534+0000","last_undegraded":"2023-12-14T18:25:00.768534+0000","last_fullsized":"2023-12-14T18:25:00.768534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:17:33.452268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:42.862743+0000","last_change":"2023-12-14T18:24:40.605099+0000","last_active":"2023-12-14T18:24:42.862743+0000","last_peered":"2023-12-14T18:24:42.862743+0000","last_clean":"2023-12-14T18:24:42.862743+0000","last_became_active":"2023-12-14T18:24:40.604773+0000","last_became_peered":"2023-12-14T18:24:40.604773+0000","last_unstale":"2023-12-14T18:24:42.862743+0000","last_undegraded":"2023-12-14T18:24:42.862743+0000","last_fullsized":"2023-12-14T18:24:42.862743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:27:44.202301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998989+0000","last_change":"2023-12-14T18:24:36.574993+0000","last_active":"2023-12-14T18:24:40.998989+0000","last_peered":"2023-12-14T18:24:40.998989+0000","last_clean":"2023-12-14T18:24:40.998989+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:24:40.998989+0000","last_undegraded":"2023-12-14T18:24:40.998989+0000","last_fullsized":"2023-12-14T18:24:40.998989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:36:49.384865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039428+0000","last_change":"2023-12-14T18:24:36.170061+0000","last_active":"2023-12-14T18:25:00.039428+0000","last_peered":"2023-12-14T18:25:00.039428+0000","last_clean":"2023-12-14T18:25:00.039428+0000","last_became_active":"2023-12-14T18:24:36.169652+0000","last_became_peered":"2023-12-14T18:24:36.169652+0000","last_unstale":"2023-12-14T18:25:00.039428+0000","last_undegraded":"2023-12-14T18:25:00.039428+0000","last_fullsized":"2023-12-14T18:25:00.039428+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:35:33.204173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941240+0000","last_change":"2023-12-14T18:23:49.076644+0000","last_active":"2023-12-14T18:24:41.941240+0000","last_peered":"2023-12-14T18:24:41.941240+0000","last_clean":"2023-12-14T18:24:41.941240+0000","last_became_active":"2023-12-14T18:23:49.076263+0000","last_became_peered":"2023-12-14T18:23:49.076263+0000","last_unstale":"2023-12-14T18:24:41.941240+0000","last_undegraded":"2023-12-14T18:24:41.941240+0000","last_fullsized":"2023-12-14T18:24:41.941240+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_clean_scrub_stamp":"2023-12-14T18:21:24.715131+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:19:15.132541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149430+0000","last_change":"2023-12-14T18:24:40.772015+0000","last_active":"2023-12-14T18:24:41.149430+0000","last_peered":"2023-12-14T18:24:41.149430+0000","last_clean":"2023-12-14T18:24:41.149430+0000","last_became_active":"2023-12-14T18:24:40.771497+0000","last_became_peered":"2023-12-14T18:24:40.771497+0000","last_unstale":"2023-12-14T18:24:41.149430+0000","last_undegraded":"2023-12-14T18:24:41.149430+0000","last_fullsized":"2023-12-14T18:24:41.149430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:37:04.368364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939552+0000","last_change":"2023-12-14T18:24:36.584402+0000","last_active":"2023-12-14T18:24:41.939552+0000","last_peered":"2023-12-14T18:24:41.939552+0000","last_clean":"2023-12-14T18:24:41.939552+0000","last_became_active":"2023-12-14T18:24:36.584177+0000","last_became_peered":"2023-12-14T18:24:36.584177+0000","last_unstale":"2023-12-14T18:24:41.939552+0000","last_undegraded":"2023-12-14T18:24:41.939552+0000","last_fullsized":"2023-12-14T18:24:41.939552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:38:45.449137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767235+0000","last_change":"2023-12-14T18:24:36.175017+0000","last_active":"2023-12-14T18:25:00.767235+0000","last_peered":"2023-12-14T18:25:00.767235+0000","last_clean":"2023-12-14T18:25:00.767235+0000","last_became_active":"2023-12-14T18:24:36.174639+0000","last_became_peered":"2023-12-14T18:24:36.174639+0000","last_unstale":"2023-12-14T18:25:00.767235+0000","last_undegraded":"2023-12-14T18:25:00.767235+0000","last_fullsized":"2023-12-14T18:25:00.767235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:56:02.780313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939585+0000","last_change":"2023-12-14T18:24:40.596043+0000","last_active":"2023-12-14T18:24:41.939585+0000","last_peered":"2023-12-14T18:24:41.939585+0000","last_clean":"2023-12-14T18:24:41.939585+0000","last_became_active":"2023-12-14T18:24:40.595603+0000","last_became_peered":"2023-12-14T18:24:40.595603+0000","last_unstale":"2023-12-14T18:24:41.939585+0000","last_undegraded":"2023-12-14T18:24:41.939585+0000","last_fullsized":"2023-12-14T18:24:41.939585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:09:47.437024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095858+0000","last_change":"2023-12-14T18:24:36.177484+0000","last_active":"2023-12-14T18:24:41.095858+0000","last_peered":"2023-12-14T18:24:41.095858+0000","last_clean":"2023-12-14T18:24:41.095858+0000","last_became_active":"2023-12-14T18:24:36.176592+0000","last_became_peered":"2023-12-14T18:24:36.176592+0000","last_unstale":"2023-12-14T18:24:41.095858+0000","last_undegraded":"2023-12-14T18:24:41.095858+0000","last_fullsized":"2023-12-14T18:24:41.095858+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:21:55.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940747+0000","last_change":"2023-12-14T18:24:36.585687+0000","last_active":"2023-12-14T18:24:41.940747+0000","last_peered":"2023-12-14T18:24:41.940747+0000","last_clean":"2023-12-14T18:24:41.940747+0000","last_became_active":"2023-12-14T18:24:36.585360+0000","last_became_peered":"2023-12-14T18:24:36.585360+0000","last_unstale":"2023-12-14T18:24:41.940747+0000","last_undegraded":"2023-12-14T18:24:41.940747+0000","last_fullsized":"2023-12-14T18:24:41.940747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:42.737826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998830+0000","last_change":"2023-12-14T18:24:40.771747+0000","last_active":"2023-12-14T18:24:40.998830+0000","last_peered":"2023-12-14T18:24:40.998830+0000","last_clean":"2023-12-14T18:24:40.998830+0000","last_became_active":"2023-12-14T18:24:40.771310+0000","last_became_peered":"2023-12-14T18:24:40.771310+0000","last_unstale":"2023-12-14T18:24:40.998830+0000","last_undegraded":"2023-12-14T18:24:40.998830+0000","last_fullsized":"2023-12-14T18:24:40.998830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:52.843438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767966+0000","last_change":"2023-12-14T18:24:36.171833+0000","last_active":"2023-12-14T18:25:00.767966+0000","last_peered":"2023-12-14T18:25:00.767966+0000","last_clean":"2023-12-14T18:25:00.767966+0000","last_became_active":"2023-12-14T18:24:36.171512+0000","last_became_peered":"2023-12-14T18:24:36.171512+0000","last_unstale":"2023-12-14T18:25:00.767966+0000","last_undegraded":"2023-12-14T18:25:00.767966+0000","last_fullsized":"2023-12-14T18:25:00.767966+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:56.553040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999452+0000","last_change":"2023-12-14T18:24:36.594938+0000","last_active":"2023-12-14T18:24:40.999452+0000","last_peered":"2023-12-14T18:24:40.999452+0000","last_clean":"2023-12-14T18:24:40.999452+0000","last_became_active":"2023-12-14T18:24:36.594573+0000","last_became_peered":"2023-12-14T18:24:36.594573+0000","last_unstale":"2023-12-14T18:24:40.999452+0000","last_undegraded":"2023-12-14T18:24:40.999452+0000","last_fullsized":"2023-12-14T18:24:40.999452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:18:30.843187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000512+0000","last_change":"2023-12-14T18:24:40.601146+0000","last_active":"2023-12-14T18:24:41.000512+0000","last_peered":"2023-12-14T18:24:41.000512+0000","last_clean":"2023-12-14T18:24:41.000512+0000","last_became_active":"2023-12-14T18:24:40.600745+0000","last_became_peered":"2023-12-14T18:24:40.600745+0000","last_unstale":"2023-12-14T18:24:41.000512+0000","last_undegraded":"2023-12-14T18:24:41.000512+0000","last_fullsized":"2023-12-14T18:24:41.000512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:48:20.003480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039339+0000","last_change":"2023-12-14T18:24:36.175880+0000","last_active":"2023-12-14T18:25:00.039339+0000","last_peered":"2023-12-14T18:25:00.039339+0000","last_clean":"2023-12-14T18:25:00.039339+0000","last_became_active":"2023-12-14T18:24:36.175514+0000","last_became_peered":"2023-12-14T18:24:36.175514+0000","last_unstale":"2023-12-14T18:25:00.039339+0000","last_undegraded":"2023-12-14T18:25:00.039339+0000","last_fullsized":"2023-12-14T18:25:00.039339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:00:40.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939608+0000","last_change":"2023-12-14T18:24:36.577062+0000","last_active":"2023-12-14T18:24:41.939608+0000","last_peered":"2023-12-14T18:24:41.939608+0000","last_clean":"2023-12-14T18:24:41.939608+0000","last_became_active":"2023-12-14T18:24:36.576733+0000","last_became_peered":"2023-12-14T18:24:36.576733+0000","last_unstale":"2023-12-14T18:24:41.939608+0000","last_undegraded":"2023-12-14T18:24:41.939608+0000","last_fullsized":"2023-12-14T18:24:41.939608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:12:08.207474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940397+0000","last_change":"2023-12-14T18:24:40.772752+0000","last_active":"2023-12-14T18:24:41.940397+0000","last_peered":"2023-12-14T18:24:41.940397+0000","last_clean":"2023-12-14T18:24:41.940397+0000","last_became_active":"2023-12-14T18:24:40.772168+0000","last_became_peered":"2023-12-14T18:24:40.772168+0000","last_unstale":"2023-12-14T18:24:41.940397+0000","last_undegraded":"2023-12-14T18:24:41.940397+0000","last_fullsized":"2023-12-14T18:24:41.940397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:02:24.129455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.038773+0000","last_change":"2023-12-14T18:24:36.178885+0000","last_active":"2023-12-14T18:25:00.038773+0000","last_peered":"2023-12-14T18:25:00.038773+0000","last_clean":"2023-12-14T18:25:00.038773+0000","last_became_active":"2023-12-14T18:24:36.178386+0000","last_became_peered":"2023-12-14T18:24:36.178386+0000","last_unstale":"2023-12-14T18:25:00.038773+0000","last_undegraded":"2023-12-14T18:25:00.038773+0000","last_fullsized":"2023-12-14T18:25:00.038773+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:37:16.134801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095247+0000","last_change":"2023-12-14T18:24:36.590037+0000","last_active":"2023-12-14T18:24:41.095247+0000","last_peered":"2023-12-14T18:24:41.095247+0000","last_clean":"2023-12-14T18:24:41.095247+0000","last_became_active":"2023-12-14T18:24:36.589659+0000","last_became_peered":"2023-12-14T18:24:36.589659+0000","last_unstale":"2023-12-14T18:24:41.095247+0000","last_undegraded":"2023-12-14T18:24:41.095247+0000","last_fullsized":"2023-12-14T18:24:41.095247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:13:57.949133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093418+0000","last_change":"2023-12-14T18:24:40.602483+0000","last_active":"2023-12-14T18:24:41.093418+0000","last_peered":"2023-12-14T18:24:41.093418+0000","last_clean":"2023-12-14T18:24:41.093418+0000","last_became_active":"2023-12-14T18:24:40.601989+0000","last_became_peered":"2023-12-14T18:24:40.601989+0000","last_unstale":"2023-12-14T18:24:41.093418+0000","last_undegraded":"2023-12-14T18:24:41.093418+0000","last_fullsized":"2023-12-14T18:24:41.093418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:09.202695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.037079+0000","last_change":"2023-12-14T18:24:36.173350+0000","last_active":"2023-12-14T18:25:00.037079+0000","last_peered":"2023-12-14T18:25:00.037079+0000","last_clean":"2023-12-14T18:25:00.037079+0000","last_became_active":"2023-12-14T18:24:36.172840+0000","last_became_peered":"2023-12-14T18:24:36.172840+0000","last_unstale":"2023-12-14T18:25:00.037079+0000","last_undegraded":"2023-12-14T18:25:00.037079+0000","last_fullsized":"2023-12-14T18:25:00.037079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:17:31.439315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939457+0000","last_change":"2023-12-14T18:24:36.592104+0000","last_active":"2023-12-14T18:24:41.939457+0000","last_peered":"2023-12-14T18:24:41.939457+0000","last_clean":"2023-12-14T18:24:41.939457+0000","last_became_active":"2023-12-14T18:24:36.591568+0000","last_became_peered":"2023-12-14T18:24:36.591568+0000","last_unstale":"2023-12-14T18:24:41.939457+0000","last_undegraded":"2023-12-14T18:24:41.939457+0000","last_fullsized":"2023-12-14T18:24:41.939457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:30.952113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940520+0000","last_change":"2023-12-14T18:24:40.596043+0000","last_active":"2023-12-14T18:24:41.940520+0000","last_peered":"2023-12-14T18:24:41.940520+0000","last_clean":"2023-12-14T18:24:41.940520+0000","last_became_active":"2023-12-14T18:24:40.595520+0000","last_became_peered":"2023-12-14T18:24:40.595520+0000","last_unstale":"2023-12-14T18:24:41.940520+0000","last_undegraded":"2023-12-14T18:24:41.940520+0000","last_fullsized":"2023-12-14T18:24:41.940520+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:45:44.615227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000728+0000","last_change":"2023-12-14T18:24:36.174184+0000","last_active":"2023-12-14T18:24:41.000728+0000","last_peered":"2023-12-14T18:24:41.000728+0000","last_clean":"2023-12-14T18:24:41.000728+0000","last_became_active":"2023-12-14T18:24:36.173850+0000","last_became_peered":"2023-12-14T18:24:36.173850+0000","last_unstale":"2023-12-14T18:24:41.000728+0000","last_undegraded":"2023-12-14T18:24:41.000728+0000","last_fullsized":"2023-12-14T18:24:41.000728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:49:58.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999206+0000","last_change":"2023-12-14T18:24:36.587777+0000","last_active":"2023-12-14T18:24:40.999206+0000","last_peered":"2023-12-14T18:24:40.999206+0000","last_clean":"2023-12-14T18:24:40.999206+0000","last_became_active":"2023-12-14T18:24:36.587317+0000","last_became_peered":"2023-12-14T18:24:36.587317+0000","last_unstale":"2023-12-14T18:24:40.999206+0000","last_undegraded":"2023-12-14T18:24:40.999206+0000","last_fullsized":"2023-12-14T18:24:40.999206+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:59:49.931215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940400+0000","last_change":"2023-12-14T18:24:40.771913+0000","last_active":"2023-12-14T18:24:41.940400+0000","last_peered":"2023-12-14T18:24:41.940400+0000","last_clean":"2023-12-14T18:24:41.940400+0000","last_became_active":"2023-12-14T18:24:40.771425+0000","last_became_peered":"2023-12-14T18:24:40.771425+0000","last_unstale":"2023-12-14T18:24:41.940400+0000","last_undegraded":"2023-12-14T18:24:41.940400+0000","last_fullsized":"2023-12-14T18:24:41.940400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:17:43.378002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039322+0000","last_change":"2023-12-14T18:24:36.176801+0000","last_active":"2023-12-14T18:25:00.039322+0000","last_peered":"2023-12-14T18:25:00.039322+0000","last_clean":"2023-12-14T18:25:00.039322+0000","last_became_active":"2023-12-14T18:24:36.176001+0000","last_became_peered":"2023-12-14T18:24:36.176001+0000","last_unstale":"2023-12-14T18:25:00.039322+0000","last_undegraded":"2023-12-14T18:25:00.039322+0000","last_fullsized":"2023-12-14T18:25:00.039322+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:16:31.583481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940941+0000","last_change":"2023-12-14T18:24:36.592898+0000","last_active":"2023-12-14T18:24:41.940941+0000","last_peered":"2023-12-14T18:24:41.940941+0000","last_clean":"2023-12-14T18:24:41.940941+0000","last_became_active":"2023-12-14T18:24:36.586650+0000","last_became_peered":"2023-12-14T18:24:36.586650+0000","last_unstale":"2023-12-14T18:24:41.940941+0000","last_undegraded":"2023-12-14T18:24:41.940941+0000","last_fullsized":"2023-12-14T18:24:41.940941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:39:25.660709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000369+0000","last_change":"2023-12-14T18:24:40.599181+0000","last_active":"2023-12-14T18:24:41.000369+0000","last_peered":"2023-12-14T18:24:41.000369+0000","last_clean":"2023-12-14T18:24:41.000369+0000","last_became_active":"2023-12-14T18:24:40.598785+0000","last_became_peered":"2023-12-14T18:24:40.598785+0000","last_unstale":"2023-12-14T18:24:41.000369+0000","last_undegraded":"2023-12-14T18:24:41.000369+0000","last_fullsized":"2023-12-14T18:24:41.000369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:11:55.342182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767511+0000","last_change":"2023-12-14T18:24:36.173644+0000","last_active":"2023-12-14T18:25:00.767511+0000","last_peered":"2023-12-14T18:25:00.767511+0000","last_clean":"2023-12-14T18:25:00.767511+0000","last_became_active":"2023-12-14T18:24:36.173059+0000","last_became_peered":"2023-12-14T18:24:36.173059+0000","last_unstale":"2023-12-14T18:25:00.767511+0000","last_undegraded":"2023-12-14T18:25:00.767511+0000","last_fullsized":"2023-12-14T18:25:00.767511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:11:21.509881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941424+0000","last_change":"2023-12-14T18:24:36.588761+0000","last_active":"2023-12-14T18:24:41.941424+0000","last_peered":"2023-12-14T18:24:41.941424+0000","last_clean":"2023-12-14T18:24:41.941424+0000","last_became_active":"2023-12-14T18:24:36.586021+0000","last_became_peered":"2023-12-14T18:24:36.586021+0000","last_unstale":"2023-12-14T18:24:41.941424+0000","last_undegraded":"2023-12-14T18:24:41.941424+0000","last_fullsized":"2023-12-14T18:24:41.941424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:23:36.358426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939203+0000","last_change":"2023-12-14T18:24:40.607286+0000","last_active":"2023-12-14T18:24:41.939203+0000","last_peered":"2023-12-14T18:24:41.939203+0000","last_clean":"2023-12-14T18:24:41.939203+0000","last_became_active":"2023-12-14T18:24:40.606925+0000","last_became_peered":"2023-12-14T18:24:40.606925+0000","last_unstale":"2023-12-14T18:24:41.939203+0000","last_undegraded":"2023-12-14T18:24:41.939203+0000","last_fullsized":"2023-12-14T18:24:41.939203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:47:39.360084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149531+0000","last_change":"2023-12-14T18:24:36.174467+0000","last_active":"2023-12-14T18:24:41.149531+0000","last_peered":"2023-12-14T18:24:41.149531+0000","last_clean":"2023-12-14T18:24:41.149531+0000","last_became_active":"2023-12-14T18:24:36.173920+0000","last_became_peered":"2023-12-14T18:24:36.173920+0000","last_unstale":"2023-12-14T18:24:41.149531+0000","last_undegraded":"2023-12-14T18:24:41.149531+0000","last_fullsized":"2023-12-14T18:24:41.149531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:36:25.392784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940237+0000","last_change":"2023-12-14T18:24:36.597537+0000","last_active":"2023-12-14T18:24:41.940237+0000","last_peered":"2023-12-14T18:24:41.940237+0000","last_clean":"2023-12-14T18:24:41.940237+0000","last_became_active":"2023-12-14T18:24:36.594599+0000","last_became_peered":"2023-12-14T18:24:36.594599+0000","last_unstale":"2023-12-14T18:24:41.940237+0000","last_undegraded":"2023-12-14T18:24:41.940237+0000","last_fullsized":"2023-12-14T18:24:41.940237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:00.181838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148562+0000","last_change":"2023-12-14T18:24:40.597990+0000","last_active":"2023-12-14T18:24:41.148562+0000","last_peered":"2023-12-14T18:24:41.148562+0000","last_clean":"2023-12-14T18:24:41.148562+0000","last_became_active":"2023-12-14T18:24:40.597568+0000","last_became_peered":"2023-12-14T18:24:40.597568+0000","last_unstale":"2023-12-14T18:24:41.148562+0000","last_undegraded":"2023-12-14T18:24:41.148562+0000","last_fullsized":"2023-12-14T18:24:41.148562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:31:38.963162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940025+0000","last_change":"2023-12-14T18:24:36.173141+0000","last_active":"2023-12-14T18:24:41.940025+0000","last_peered":"2023-12-14T18:24:41.940025+0000","last_clean":"2023-12-14T18:24:41.940025+0000","last_became_active":"2023-12-14T18:24:36.172811+0000","last_became_peered":"2023-12-14T18:24:36.172811+0000","last_unstale":"2023-12-14T18:24:41.940025+0000","last_undegraded":"2023-12-14T18:24:41.940025+0000","last_fullsized":"2023-12-14T18:24:41.940025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:01:41.375107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941679+0000","last_change":"2023-12-14T18:24:36.587333+0000","last_active":"2023-12-14T18:24:41.941679+0000","last_peered":"2023-12-14T18:24:41.941679+0000","last_clean":"2023-12-14T18:24:41.941679+0000","last_became_active":"2023-12-14T18:24:36.586914+0000","last_became_peered":"2023-12-14T18:24:36.586914+0000","last_unstale":"2023-12-14T18:24:41.941679+0000","last_undegraded":"2023-12-14T18:24:41.941679+0000","last_fullsized":"2023-12-14T18:24:41.941679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:44:34.232245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939625+0000","last_change":"2023-12-14T18:24:36.591896+0000","last_active":"2023-12-14T18:24:41.939625+0000","last_peered":"2023-12-14T18:24:41.939625+0000","last_clean":"2023-12-14T18:24:41.939625+0000","last_became_active":"2023-12-14T18:24:36.591097+0000","last_became_peered":"2023-12-14T18:24:36.591097+0000","last_unstale":"2023-12-14T18:24:41.939625+0000","last_undegraded":"2023-12-14T18:24:41.939625+0000","last_fullsized":"2023-12-14T18:24:41.939625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:14:56.435755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000778+0000","last_change":"2023-12-14T18:24:36.581688+0000","last_active":"2023-12-14T18:24:41.000778+0000","last_peered":"2023-12-14T18:24:41.000778+0000","last_clean":"2023-12-14T18:24:41.000778+0000","last_became_active":"2023-12-14T18:24:36.580660+0000","last_became_peered":"2023-12-14T18:24:36.580660+0000","last_unstale":"2023-12-14T18:24:41.000778+0000","last_undegraded":"2023-12-14T18:24:41.000778+0000","last_fullsized":"2023-12-14T18:24:41.000778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:13:48.827412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000650+0000","last_change":"2023-12-14T18:24:36.583025+0000","last_active":"2023-12-14T18:24:41.000650+0000","last_peered":"2023-12-14T18:24:41.000650+0000","last_clean":"2023-12-14T18:24:41.000650+0000","last_became_active":"2023-12-14T18:24:36.582510+0000","last_became_peered":"2023-12-14T18:24:36.582510+0000","last_unstale":"2023-12-14T18:24:41.000650+0000","last_undegraded":"2023-12-14T18:24:41.000650+0000","last_fullsized":"2023-12-14T18:24:41.000650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:21:38.503753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939373+0000","last_change":"2023-12-14T18:24:36.579383+0000","last_active":"2023-12-14T18:24:41.939373+0000","last_peered":"2023-12-14T18:24:41.939373+0000","last_clean":"2023-12-14T18:24:41.939373+0000","last_became_active":"2023-12-14T18:24:36.579037+0000","last_became_peered":"2023-12-14T18:24:36.579037+0000","last_unstale":"2023-12-14T18:24:41.939373+0000","last_undegraded":"2023-12-14T18:24:41.939373+0000","last_fullsized":"2023-12-14T18:24:41.939373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:13:32.810036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940749+0000","last_change":"2023-12-14T18:24:36.582940+0000","last_active":"2023-12-14T18:24:41.940749+0000","last_peered":"2023-12-14T18:24:41.940749+0000","last_clean":"2023-12-14T18:24:41.940749+0000","last_became_active":"2023-12-14T18:24:36.582645+0000","last_became_peered":"2023-12-14T18:24:36.582645+0000","last_unstale":"2023-12-14T18:24:41.940749+0000","last_undegraded":"2023-12-14T18:24:41.940749+0000","last_fullsized":"2023-12-14T18:24:41.940749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:43:46.761797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940015+0000","last_change":"2023-12-14T18:24:36.578613+0000","last_active":"2023-12-14T18:24:41.940015+0000","last_peered":"2023-12-14T18:24:41.940015+0000","last_clean":"2023-12-14T18:24:41.940015+0000","last_became_active":"2023-12-14T18:24:36.578287+0000","last_became_peered":"2023-12-14T18:24:36.578287+0000","last_unstale":"2023-12-14T18:24:41.940015+0000","last_undegraded":"2023-12-14T18:24:41.940015+0000","last_fullsized":"2023-12-14T18:24:41.940015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:05:43.072668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001258+0000","last_change":"2023-12-14T18:24:36.575078+0000","last_active":"2023-12-14T18:24:41.001258+0000","last_peered":"2023-12-14T18:24:41.001258+0000","last_clean":"2023-12-14T18:24:41.001258+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:24:41.001258+0000","last_undegraded":"2023-12-14T18:24:41.001258+0000","last_fullsized":"2023-12-14T18:24:41.001258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:55:31.886500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999571+0000","last_change":"2023-12-14T18:24:36.572545+0000","last_active":"2023-12-14T18:24:40.999571+0000","last_peered":"2023-12-14T18:24:40.999571+0000","last_clean":"2023-12-14T18:24:40.999571+0000","last_became_active":"2023-12-14T18:24:36.572235+0000","last_became_peered":"2023-12-14T18:24:36.572235+0000","last_unstale":"2023-12-14T18:24:40.999571+0000","last_undegraded":"2023-12-14T18:24:40.999571+0000","last_fullsized":"2023-12-14T18:24:40.999571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:25:56.081431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148685+0000","last_change":"2023-12-14T18:24:36.588278+0000","last_active":"2023-12-14T18:24:41.148685+0000","last_peered":"2023-12-14T18:24:41.148685+0000","last_clean":"2023-12-14T18:24:41.148685+0000","last_became_active":"2023-12-14T18:24:36.587605+0000","last_became_peered":"2023-12-14T18:24:36.587605+0000","last_unstale":"2023-12-14T18:24:41.148685+0000","last_undegraded":"2023-12-14T18:24:41.148685+0000","last_fullsized":"2023-12-14T18:24:41.148685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:32:11.935687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998910+0000","last_change":"2023-12-14T18:24:36.575325+0000","last_active":"2023-12-14T18:24:40.998910+0000","last_peered":"2023-12-14T18:24:40.998910+0000","last_clean":"2023-12-14T18:24:40.998910+0000","last_became_active":"2023-12-14T18:24:36.575016+0000","last_became_peered":"2023-12-14T18:24:36.575016+0000","last_unstale":"2023-12-14T18:24:40.998910+0000","last_undegraded":"2023-12-14T18:24:40.998910+0000","last_fullsized":"2023-12-14T18:24:40.998910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:35:26.394701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000615+0000","last_change":"2023-12-14T18:24:36.585348+0000","last_active":"2023-12-14T18:24:41.000615+0000","last_peered":"2023-12-14T18:24:41.000615+0000","last_clean":"2023-12-14T18:24:41.000615+0000","last_became_active":"2023-12-14T18:24:36.584995+0000","last_became_peered":"2023-12-14T18:24:36.584995+0000","last_unstale":"2023-12-14T18:24:41.000615+0000","last_undegraded":"2023-12-14T18:24:41.000615+0000","last_fullsized":"2023-12-14T18:24:41.000615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:22:10.788069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939979+0000","last_change":"2023-12-14T18:24:36.583571+0000","last_active":"2023-12-14T18:24:41.939979+0000","last_peered":"2023-12-14T18:24:41.939979+0000","last_clean":"2023-12-14T18:24:41.939979+0000","last_became_active":"2023-12-14T18:24:36.583266+0000","last_became_peered":"2023-12-14T18:24:36.583266+0000","last_unstale":"2023-12-14T18:24:41.939979+0000","last_undegraded":"2023-12-14T18:24:41.939979+0000","last_fullsized":"2023-12-14T18:24:41.939979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:09:22.877080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939741+0000","last_change":"2023-12-14T18:24:36.579754+0000","last_active":"2023-12-14T18:24:41.939741+0000","last_peered":"2023-12-14T18:24:41.939741+0000","last_clean":"2023-12-14T18:24:41.939741+0000","last_became_active":"2023-12-14T18:24:36.579337+0000","last_became_peered":"2023-12-14T18:24:36.579337+0000","last_unstale":"2023-12-14T18:24:41.939741+0000","last_undegraded":"2023-12-14T18:24:41.939741+0000","last_fullsized":"2023-12-14T18:24:41.939741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:48:45.306297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":12288,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1,"ondisk_log_size":1,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212533,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":6621,"num_objects":52,"num_object_clones":0,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":117,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":36,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":237568,"data_stored":54766,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":100,"ondisk_log_size":100,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547214,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6016,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5120,"kb_avail":93739136,"statfs":{"total":95995035648,"available":95988875264,"internally_reserved":0,"allocated":909312,"data_stored":453962,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5242880},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64400000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48299999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60599999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57899999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51200000000000001}]},{"osd":9,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67600000000000005}]},{"osd":10,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":10,"up_from":67,"seq":287762808850,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6288,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738864,"statfs":{"total":95995035648,"available":95988596736,"internally_reserved":0,"allocated":991232,"data_stored":1048748,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46700000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.6690000000000005,"5min":8.6690000000000005,"15min":8.6690000000000005},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.59299999999999997},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":9.5459999999999994,"5min":9.5459999999999994,"15min":9.5459999999999994},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":9.4380000000000006,"5min":9.4380000000000006,"15min":9.4380000000000006},"last":0.504},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.48899999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.9470000000000001,"5min":5.9470000000000001,"15min":5.9470000000000001},"last":0.443},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":6.1470000000000002,"5min":6.1470000000000002,"15min":6.1470000000000002},"last":0.39500000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.67200000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":9.4830000000000005,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":0.629},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.64600000000000002}]},{"osd":9,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.70599999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":9,"up_from":62,"seq":266287972373,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6232,"kb_used_data":912,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738920,"statfs":{"total":95995035648,"available":95988654080,"internally_reserved":0,"allocated":933888,"data_stored":458876,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.27200000000000002},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.433}]},{"osd":2,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.623,"5min":0.623,"15min":0.623},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.52700000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.497,"5min":0.497,"15min":0.497},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.60699999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.64200000000000002,"15min":0.64200000000000002},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.64700000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.69999999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.66900000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.71499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68200000000000005}]}]},{"osd":8,"up_from":55,"seq":236223201305,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6280,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738872,"statfs":{"total":95995035648,"available":95988604928,"internally_reserved":0,"allocated":917504,"data_stored":454257,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.625}]},{"osd":1,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.64500000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.69199999999999995}]},{"osd":3,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.77400000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"last":0.84599999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.76300000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":1.042}]},{"osd":7,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.872,"5min":0.872,"15min":0.872},"last":0.94999999999999996},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.872}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.90600000000000003},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":1.016}]},{"osd":10,"last update":"Thu Dec 14 18:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.92400000000000004}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81000000000000005}]}]},{"osd":7,"up_from":48,"seq":206158430236,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6464,"kb_used_data":952,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738688,"statfs":{"total":95995035648,"available":95988416512,"internally_reserved":0,"allocated":974848,"data_stored":1050181,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.434}]},{"osd":1,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.40699999999999997}]},{"osd":2,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.57999999999999996}]},{"osd":3,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56699999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.53600000000000003,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.7830000000000004,"5min":5.7830000000000004,"15min":5.7830000000000004},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.44700000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.63},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.68300000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.52000000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77500000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.54400000000000004}]},{"osd":8,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.67800000000000005,"15min":0.67800000000000005},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.879,"5min":0.879,"15min":0.879},"last":0.71199999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.72799999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.75600000000000001}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77600000000000002}]}]},{"osd":6,"up_from":43,"seq":184683593758,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6272,"kb_used_data":888,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738880,"statfs":{"total":95995035648,"available":95988613120,"internally_reserved":0,"allocated":909312,"data_stored":453962,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38500000000000001},"min":{"1min":0.23400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.253,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.37}]},{"osd":1,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70399999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.629}]},{"osd":2,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.745,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.497}]},{"osd":3,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70899999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.46400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.55500000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60499999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.624,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.68600000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:24:12 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.54400000000000004}]},{"osd":9,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.85299999999999998,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.47199999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.72899999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7180,"kb_used_data":900,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737972,"statfs":{"total":95995035648,"available":95987683328,"internally_reserved":0,"allocated":921600,"data_stored":457755,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.49199999999999999,"5min":0.47099999999999997,"15min":0.45800000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.5,"5min":0.47599999999999998,"15min":0.45800000000000002},"min":{"1min":0.32100000000000001,"5min":0.314,"15min":0.314},"max":{"1min":0.65900000000000003,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.46100000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.498,"15min":0.5},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.71799999999999997}]},{"osd":3,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47799999999999998,"15min":0.46500000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.74199999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:24:03 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.41099999999999998,"15min":0.40300000000000002},"min":{"1min":0.29499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.40799999999999997,"15min":0.39800000000000002},"min":{"1min":0.28499999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":1.542,"5min":1.542,"15min":1.542},"last":0.61299999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:24:20 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.91200000000000003,"5min":0.91200000000000003,"15min":0.91200000000000003},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.54300000000000004,"15min":0.54200000000000004},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.752,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.76400000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.70099999999999996}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.79100000000000004},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83299999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.73099999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.753},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.83799999999999997},"last":0.80500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64100000000000001,"15min":0.64100000000000001},"min":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.88600000000000001,"5min":0.88600000000000001,"15min":0.88600000000000001},"last":0.77800000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.86899999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81799999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738421,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7172,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737980,"statfs":{"total":95995035648,"available":95987691520,"internally_reserved":0,"allocated":913408,"data_stored":454052,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47499999999999998,"15min":0.47399999999999998},"min":{"1min":0.27200000000000002,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.60399999999999998,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.47899999999999998,"15min":0.47299999999999998},"min":{"1min":0.29299999999999998,"5min":0.222,"15min":0.222},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.67900000000000005}]},{"osd":2,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.5,"15min":0.48799999999999999},"min":{"1min":0.36499999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.78800000000000003,"5min":0.79200000000000004,"15min":0.79200000000000004},"last":0.95199999999999996},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.51900000000000002,"15min":0.49299999999999999},"min":{"1min":0.41299999999999998,"5min":0.373,"15min":0.373},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.81799999999999995}]},{"osd":3,"last update":"Thu Dec 14 18:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.27300000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35499999999999998},"min":{"1min":0.26700000000000002,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:24:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.50600000000000001,"15min":0.498},"min":{"1min":0.39000000000000001,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.502,"15min":0.498},"min":{"1min":0.36899999999999999,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.71799999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.48899999999999999,"15min":0.47799999999999998},"min":{"1min":0.36399999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.51900000000000002,"15min":0.50700000000000001},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"last":0.91200000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.76000000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.88800000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.93500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"last":0.86699999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.83299999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.99099999999999999},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":1.0569999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97099999999999997}]}]},{"osd":2,"up_from":18,"seq":77309411374,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7368,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737784,"statfs":{"total":95995035648,"available":95987490816,"internally_reserved":0,"allocated":983040,"data_stored":494168,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.434},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.64200000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.44900000000000001},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.499,"15min":0.47099999999999997},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.51500000000000001,"15min":0.48599999999999999},"min":{"1min":0.39500000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.85599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.79200000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.46600000000000003},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76100000000000001,"5min":1.179,"15min":1.179},"last":0.50600000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.39700000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.437,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.66100000000000003}]},{"osd":5,"last update":"Thu Dec 14 18:24:19 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.47399999999999998,"15min":0.47199999999999998},"min":{"1min":0.32800000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.67600000000000005,"5min":0.73699999999999999,"15min":0.73699999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.51400000000000001,"15min":0.51100000000000001},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"last":0.36699999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59899999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.71899999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.38500000000000001,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.623}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":0.90100000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.55500000000000005}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63900000000000001}]}]},{"osd":3,"up_from":24,"seq":103079215146,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":925696,"data_stored":454246,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.32800000000000001},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.248,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.29899999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.497,"15min":0.47499999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.45800000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.55000000000000004}]},{"osd":4,"last update":"Thu Dec 14 18:24:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.42399999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.214,"5min":1.214,"15min":1.214},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.57799999999999996,"15min":0.57099999999999995},"min":{"1min":0.48399999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76500000000000001,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.52300000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.55100000000000005,"15min":0.54300000000000004},"min":{"1min":0.40400000000000003,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.73299999999999998,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.49199999999999999,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.497}]},{"osd":6,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.497,"15min":0.499},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":0.755},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.36799999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:24:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.628},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.48399999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:24:28 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.316},{"interface":"front","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.35799999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.627}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51400000000000001}]}]},{"osd":4,"up_from":31,"seq":133143986215,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6404,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738748,"statfs":{"total":95995035648,"available":95988477952,"internally_reserved":0,"allocated":913408,"data_stored":453984,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48799999999999999,"15min":0.48099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.372,"15min":0.372},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.41699999999999998,"5min":0.39000000000000001,"15min":0.38600000000000001},"min":{"1min":0.27300000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.53800000000000003,"15min":0.53000000000000003},"min":{"1min":0.45800000000000002,"5min":0.375,"15min":0.375},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.54300000000000004,"15min":0.53900000000000003},"min":{"1min":0.42399999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.748,"5min":0.748,"15min":0.748},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.51100000000000001,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.35099999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.67100000000000004,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.60899999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:24:24 2023","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.57999999999999996,"15min":0.57399999999999995},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.58099999999999996,"15min":0.57699999999999996},"min":{"1min":0.45200000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.54600000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.498}]},{"osd":7,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.46600000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:24:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.65600000000000003,"15min":0.65600000000000003},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63600000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.59799999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.622}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822691,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6604,"kb_used_data":964,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738548,"statfs":{"total":95995035648,"available":95988273152,"internally_reserved":0,"allocated":987136,"data_stored":1045758,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.38600000000000001,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.79300000000000004,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.433},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.52200000000000002,"15min":0.51200000000000001},"min":{"1min":0.40799999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53400000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.53900000000000003,"15min":0.53100000000000003},"min":{"1min":0.47199999999999998,"5min":0.311,"15min":0.311},"max":{"1min":0.77500000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54700000000000004,"15min":0.54400000000000004},"min":{"1min":0.45500000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"last":0.70599999999999996}]},{"osd":2,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.41699999999999998,"15min":0.41399999999999998},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.40300000000000002,"15min":0.39900000000000002},"min":{"1min":0.28299999999999997,"5min":0.248,"15min":0.248},"max":{"1min":0.69499999999999995,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.55000000000000004,"15min":0.54100000000000004},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.54700000000000004,"15min":0.53500000000000003},"min":{"1min":0.39400000000000002,"5min":0.378,"15min":0.378},"max":{"1min":0.85899999999999999,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:24:23 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.57599999999999996,"15min":0.56299999999999994},"min":{"1min":0.47699999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57999999999999996,"15min":0.57499999999999996},"min":{"1min":0.41399999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.73999999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.997,"15min":0.997},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.72399999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:24:10 2023","interfaces":[{"interface":"back","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.98499999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.92900000000000005,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.66300000000000003}]},{"osd":9,"last update":"Thu Dec 14 18:24:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.629},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.64000000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.77500000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":3793,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":40206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":626,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":5323,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:25:04.796 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-14T18:25:04.797 INFO:tasks.cephadm:Setting up client nodes... 2023-12-14T18:25:04.797 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2023-12-14T18:25:05.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:05 smithi093 ceph-mon[42070]: pgmap v238: 129 pgs: 129 active+clean; 583 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-12-14T18:25:05.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:05 smithi093 ceph-mon[42070]: mgrmap e17: x(active, since 5m), standbys: z, y 2023-12-14T18:25:05.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:05 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4265482339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:05.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:05 smithi093 ceph-mon[42070]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:05.142 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:05 smithi097 ceph-mon[43486]: pgmap v238: 129 pgs: 129 active+clean; 583 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-12-14T18:25:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:05 smithi097 ceph-mon[43486]: mgrmap e17: x(active, since 5m), standbys: z, y 2023-12-14T18:25:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:05 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4265482339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:05 smithi097 ceph-mon[43486]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:05.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:05 smithi160 ceph-mon[43840]: pgmap v238: 129 pgs: 129 active+clean; 583 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 1.3 KiB/s wr, 7 op/s 2023-12-14T18:25:05.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:05 smithi160 ceph-mon[43840]: mgrmap e17: x(active, since 5m), standbys: z, y 2023-12-14T18:25:05.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:05 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4265482339' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:05.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:05 smithi160 ceph-mon[43840]: from='client.14913 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.067+0000 7f7362aac700 1 -- 172.21.15.93:0/2098362358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 msgr2=0x7f735c100830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.067+0000 7f7362aac700 1 --2- 172.21.15.93:0/2098362358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 0x7f735c100830 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f7348009990 tx=0x7f7348009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.067+0000 7f7362aac700 1 -- 172.21.15.93:0/2098362358 shutdown_connections 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.067+0000 7f7362aac700 1 --2- 172.21.15.93:0/2098362358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c100d70 0x7f735c107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.067+0000 7f7362aac700 1 --2- 172.21.15.93:0/2098362358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f735c0ff1e0 0x7f735c0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.067+0000 7f7362aac700 1 --2- 172.21.15.93:0/2098362358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 0x7f735c100830 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.067+0000 7f7362aac700 1 -- 172.21.15.93:0/2098362358 >> 172.21.15.93:0/2098362358 conn(0x7f735c0fa7f0 msgr2=0x7f735c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.067+0000 7f7362aac700 1 -- 172.21.15.93:0/2098362358 shutdown_connections 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.067+0000 7f7362aac700 1 -- 172.21.15.93:0/2098362358 wait complete. 2023-12-14T18:25:06.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7362aac700 1 Processor -- start 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7362aac700 1 -- start start 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7362aac700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f735c0ff1e0 0x7f735c194ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7362aac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 0x7f735c195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7362aac700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c100d70 0x7f735c19c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7362aac700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c19ca80 con 0x7f735c1003e0 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7362aac700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c19cbc0 con 0x7f735c0ff1e0 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7362aac700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f735c19cd50 con 0x7f735c100d70 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f735bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 0x7f735c195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f735bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 0x7f735c195430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56944/0 (socket says 172.21.15.93:56944) 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f735bfff700 1 -- 172.21.15.93:0/187494685 learned_addr learned my addr 172.21.15.93:0/187494685 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7360848700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f735c0ff1e0 0x7f735c194ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f7361049700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c100d70 0x7f735c19c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f735bfff700 1 -- 172.21.15.93:0/187494685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c100d70 msgr2=0x7f735c19c4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f735bfff700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c100d70 0x7f735c19c4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f735bfff700 1 -- 172.21.15.93:0/187494685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f735c0ff1e0 msgr2=0x7f735c194ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:06.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f735bfff700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f735c0ff1e0 0x7f735c194ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.068+0000 7f735bfff700 1 -- 172.21.15.93:0/187494685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73480095f0 con 0x7f735c1003e0 2023-12-14T18:25:06.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.069+0000 7f735bfff700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 0x7f735c195430 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7348005910 tx=0x7f734800f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:06.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.069+0000 7f7359ffb700 1 -- 172.21.15.93:0/187494685 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734801d070 con 0x7f735c1003e0 2023-12-14T18:25:06.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.069+0000 7f7359ffb700 1 -- 172.21.15.93:0/187494685 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7348022470 con 0x7f735c1003e0 2023-12-14T18:25:06.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.069+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f735c10ab40 con 0x7f735c1003e0 2023-12-14T18:25:06.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.069+0000 7f7359ffb700 1 -- 172.21.15.93:0/187494685 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7348017650 con 0x7f735c1003e0 2023-12-14T18:25:06.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.069+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f735c10a9b0 con 0x7f735c1003e0 2023-12-14T18:25:06.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.070+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f735c1067d0 con 0x7f735c1003e0 2023-12-14T18:25:06.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.071+0000 7f7359ffb700 1 -- 172.21.15.93:0/187494685 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f73480225e0 con 0x7f735c1003e0 2023-12-14T18:25:06.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.071+0000 7f7359ffb700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f734c0a11e0 0x7f734c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:06.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.072+0000 7f7360848700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f734c0a11e0 0x7f734c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:06.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.072+0000 7f7359ffb700 1 -- 172.21.15.93:0/187494685 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f73480cdf10 con 0x7f735c1003e0 2023-12-14T18:25:06.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.072+0000 7f7360848700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f734c0a11e0 0x7f734c0a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f735c100240 tx=0x7f7350007400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:06.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.076+0000 7f7359ffb700 1 -- 172.21.15.93:0/187494685 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f73480973a0 con 0x7f735c1003e0 2023-12-14T18:25:06.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:06 smithi093 ceph-mon[42070]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:06.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:06 smithi097 ceph-mon[43486]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:06.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.348+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f735c061700 con 0x7f735c1003e0 2023-12-14T18:25:06.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.352+0000 7f7359ffb700 1 -- 172.21.15.93:0/187494685 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v31) v1 ==== 170+0+59 (secure 0 0 0) 0x7f73480270f0 con 0x7f735c1003e0 2023-12-14T18:25:06.353 INFO:teuthology.orchestra.run.smithi093.stdout:[client.0] 2023-12-14T18:25:06.353 INFO:teuthology.orchestra.run.smithi093.stdout: key = AQCCSHtl0YvKFBAAFwQE/sOGUaSJld9ZxvjtQw== 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f734c0a11e0 msgr2=0x7f734c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f734c0a11e0 0x7f734c0a3690 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f735c100240 tx=0x7f7350007400 comp rx=0 tx=0).stop 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 msgr2=0x7f735c195430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 0x7f735c195430 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7348005910 tx=0x7f734800f710 comp rx=0 tx=0).stop 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 shutdown_connections 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f734c0a11e0 0x7f734c0a3690 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f735c100d70 0x7f735c19c4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f735c0ff1e0 0x7f735c194ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 --2- 172.21.15.93:0/187494685 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f735c1003e0 0x7f735c195430 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 >> 172.21.15.93:0/187494685 conn(0x7f735c0fa7f0 msgr2=0x7f735c103910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:06.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 shutdown_connections 2023-12-14T18:25:06.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:06.354+0000 7f7362aac700 1 -- 172.21.15.93:0/187494685 wait complete. 2023-12-14T18:25:06.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:06 smithi160 ceph-mon[43840]: from='client.14919 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:06.726 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:25:06.726 DEBUG:teuthology.orchestra.run.smithi093:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2023-12-14T18:25:06.726 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-14T18:25:06.767 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-14T18:25:06.768 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2023-12-14T18:25:06.768 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph mgr dump --format=json 2023-12-14T18:25:07.119 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:07.292 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:07 smithi093 ceph-mon[42070]: pgmap v239: 129 pgs: 129 active+clean; 584 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-12-14T18:25:07.292 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:07 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/187494685' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-14T18:25:07.292 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:07 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/187494685' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-14T18:25:07.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:07 smithi097 ceph-mon[43486]: pgmap v239: 129 pgs: 129 active+clean; 584 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-12-14T18:25:07.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:07 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/187494685' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-14T18:25:07.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:07 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/187494685' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-14T18:25:07.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:07 smithi160 ceph-mon[43840]: pgmap v239: 129 pgs: 129 active+clean; 584 KiB data, 77 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 2.2 KiB/s wr, 9 op/s 2023-12-14T18:25:07.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:07 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/187494685' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2023-12-14T18:25:07.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:07 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/187494685' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2023-12-14T18:25:08.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.114+0000 7f117348a700 1 -- 172.21.15.93:0/389279969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c104510 msgr2=0x7f116c10c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:08.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.114+0000 7f117348a700 1 --2- 172.21.15.93:0/389279969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c104510 0x7f116c10c090 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f116400b0d0 tx=0x7f116400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.114+0000 7f117348a700 1 -- 172.21.15.93:0/389279969 shutdown_connections 2023-12-14T18:25:08.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.114+0000 7f117348a700 1 --2- 172.21.15.93:0/389279969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c104510 0x7f116c10c090 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.114+0000 7f117348a700 1 --2- 172.21.15.93:0/389279969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0ff470 0x7f116c0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.114+0000 7f117348a700 1 --2- 172.21.15.93:0/389279969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f116c0ffe10 0x7f116c103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.114+0000 7f117348a700 1 -- 172.21.15.93:0/389279969 >> 172.21.15.93:0/389279969 conn(0x7f116c0770d0 msgr2=0x7f116c0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:08.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.114+0000 7f117348a700 1 -- 172.21.15.93:0/389279969 shutdown_connections 2023-12-14T18:25:08.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.114+0000 7f117348a700 1 -- 172.21.15.93:0/389279969 wait complete. 2023-12-14T18:25:08.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f117348a700 1 Processor -- start 2023-12-14T18:25:08.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f117348a700 1 -- start start 2023-12-14T18:25:08.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f117348a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0ff470 0x7f116c195330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:08.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f117348a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c0ffe10 0x7f116c195870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:08.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f117348a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f116c104510 0x7f116c199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:08.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f117348a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c19a2f0 con 0x7f116c0ffe10 2023-12-14T18:25:08.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f117348a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c19a460 con 0x7f116c0ff470 2023-12-14T18:25:08.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f117348a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f116c19a620 con 0x7f116c104510 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1170a25700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c0ffe10 0x7f116c195870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1170a25700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c0ffe10 0x7f116c195870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56956/0 (socket says 172.21.15.93:56956) 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1170a25700 1 -- 172.21.15.93:0/3483843258 learned_addr learned my addr 172.21.15.93:0/3483843258 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1171226700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0ff470 0x7f116c195330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1171a27700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f116c104510 0x7f116c199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1170a25700 1 -- 172.21.15.93:0/3483843258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f116c104510 msgr2=0x7f116c199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1170a25700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f116c104510 0x7f116c199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1170a25700 1 -- 172.21.15.93:0/3483843258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0ff470 msgr2=0x7f116c195330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1170a25700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0ff470 0x7f116c195330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.115+0000 7f1170a25700 1 -- 172.21.15.93:0/3483843258 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1164009d00 con 0x7f116c0ffe10 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.116+0000 7f1170a25700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c0ffe10 0x7f116c195870 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f116800d980 tx=0x7f116800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.116+0000 7f11627fc700 1 -- 172.21.15.93:0/3483843258 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800f890 con 0x7f116c0ffe10 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.116+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f116c10f270 con 0x7f116c0ffe10 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.116+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f116c10f0e0 con 0x7f116c0ffe10 2023-12-14T18:25:08.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.116+0000 7f11627fc700 1 -- 172.21.15.93:0/3483843258 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1168010460 con 0x7f116c0ffe10 2023-12-14T18:25:08.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.116+0000 7f11627fc700 1 -- 172.21.15.93:0/3483843258 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f116800e610 con 0x7f116c0ffe10 2023-12-14T18:25:08.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.117+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1150005320 con 0x7f116c0ffe10 2023-12-14T18:25:08.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.118+0000 7f11627fc700 1 -- 172.21.15.93:0/3483843258 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f116800f9f0 con 0x7f116c0ffe10 2023-12-14T18:25:08.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.118+0000 7f11627fc700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f11580a1260 0x7f11580a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:08.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.119+0000 7f1171226700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f11580a1260 0x7f11580a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:08.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.119+0000 7f11627fc700 1 -- 172.21.15.93:0/3483843258 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f11680cc840 con 0x7f116c0ffe10 2023-12-14T18:25:08.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.119+0000 7f1171226700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f11580a1260 0x7f11580a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f115c00ac50 tx=0x7f115c00a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:08.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.123+0000 7f11627fc700 1 -- 172.21.15.93:0/3483843258 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f11680996d0 con 0x7f116c0ffe10 2023-12-14T18:25:08.369 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.368+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f1150005cc0 con 0x7f116c0ffe10 2023-12-14T18:25:08.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.373+0000 7f11627fc700 1 -- 172.21.15.93:0/3483843258 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+260124 (secure 0 0 0) 0x7f1168099260 con 0x7f116c0ffe10 2023-12-14T18:25:08.376 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:08.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.375+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f11580a1260 msgr2=0x7f11580a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:08.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.375+0000 7f117348a700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f11580a1260 0x7f11580a3710 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f115c00ac50 tx=0x7f115c00a380 comp rx=0 tx=0).stop 2023-12-14T18:25:08.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.375+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c0ffe10 msgr2=0x7f116c195870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:08.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.375+0000 7f117348a700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c0ffe10 0x7f116c195870 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f116800d980 tx=0x7f116800dc90 comp rx=0 tx=0).stop 2023-12-14T18:25:08.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.376+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 shutdown_connections 2023-12-14T18:25:08.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.376+0000 7f117348a700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f11580a1260 0x7f11580a3710 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.376+0000 7f117348a700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f116c104510 0x7f116c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.378 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.376+0000 7f117348a700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f116c0ff470 0x7f116c195330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.376+0000 7f117348a700 1 --2- 172.21.15.93:0/3483843258 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f116c0ffe10 0x7f116c195870 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:08.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.376+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 >> 172.21.15.93:0/3483843258 conn(0x7f116c0770d0 msgr2=0x7f116c0fcc10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:08.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.376+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 shutdown_connections 2023-12-14T18:25:08.379 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:08.376+0000 7f117348a700 1 -- 172.21.15.93:0/3483843258 wait complete. 2023-12-14T18:25:08.792 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":17,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6800","nonce":3802499407},{"type":"v1","addr":"172.21.15.93:6801","nonce":3802499407}]},"active_addr":"172.21.15.93:6801/3802499407","active_change":"2023-12-14T18:19:35.884573+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":24101,"name":"z","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24107,"name":"y","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_password":{"name":"alertmanager_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web password","long_desc":"","tags":[],"see_also":[]},"alertmanager_web_user":{"name":"alertmanager_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Alertmanager web user","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"prometheus_web_password":{"name":"prometheus_web_password","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web password","long_desc":"","tags":[],"see_also":[]},"prometheus_web_user":{"name":"prometheus_web_user","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"Prometheus web user","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"noautoscale":{"name":"noautoscale","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"global autoscale flag","long_desc":"Option to turn on/off the autoscaler for all pools","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.93:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":1420703523}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":1584909638}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":978480379}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":932543927}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.93:0","nonce":2319253389}]}]} 2023-12-14T18:25:08.796 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2023-12-14T18:25:08.796 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2023-12-14T18:25:08.797 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd dump --format=json 2023-12-14T18:25:09.109 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:09.312 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:09 smithi093 ceph-mon[42070]: pgmap v240: 129 pgs: 129 active+clean; 618 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-12-14T18:25:09.312 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:09 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3483843258' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-14T18:25:09.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:09 smithi097 ceph-mon[43486]: pgmap v240: 129 pgs: 129 active+clean; 618 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-12-14T18:25:09.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:09 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3483843258' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-14T18:25:09.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:09 smithi160 ceph-mon[43840]: pgmap v240: 129 pgs: 129 active+clean; 618 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-12-14T18:25:09.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:09 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3483843258' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.172+0000 7f8804215700 1 -- 172.21.15.93:0/3541960037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100ee0 msgr2=0x7f87fc104eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.172+0000 7f8804215700 1 --2- 172.21.15.93:0/3541960037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100ee0 0x7f87fc104eb0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f87f80099c0 tx=0x7f87f8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.172+0000 7f8804215700 1 -- 172.21.15.93:0/3541960037 shutdown_connections 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.172+0000 7f8804215700 1 --2- 172.21.15.93:0/3541960037 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87fc1055e0 0x7f87fc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.172+0000 7f8804215700 1 --2- 172.21.15.93:0/3541960037 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc100540 0x7f87fc100910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.172+0000 7f8804215700 1 --2- 172.21.15.93:0/3541960037 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100ee0 0x7f87fc104eb0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.172+0000 7f8804215700 1 -- 172.21.15.93:0/3541960037 >> 172.21.15.93:0/3541960037 conn(0x7f87fc0fbdc0 msgr2=0x7f87fc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.172+0000 7f8804215700 1 -- 172.21.15.93:0/3541960037 shutdown_connections 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.172+0000 7f8804215700 1 -- 172.21.15.93:0/3541960037 wait complete. 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.173+0000 7f8804215700 1 Processor -- start 2023-12-14T18:25:10.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.173+0000 7f8804215700 1 -- start start 2023-12-14T18:25:10.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.173+0000 7f8804215700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100540 0x7f87fc196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:10.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.173+0000 7f8804215700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc100ee0 0x7f87fc196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:10.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.173+0000 7f8804215700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87fc1055e0 0x7f87fc19acf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:10.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.173+0000 7f8804215700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc19b300 con 0x7f87fc100540 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.173+0000 7f8804215700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc19b470 con 0x7f87fc1055e0 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.173+0000 7f8804215700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87fc19b630 con 0x7f87fc100ee0 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.173+0000 7f8801fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100540 0x7f87fc196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8801fb1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100540 0x7f87fc196340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56976/0 (socket says 172.21.15.93:56976) 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8801fb1700 1 -- 172.21.15.93:0/3490169839 learned_addr learned my addr 172.21.15.93:0/3490169839 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f88027b2700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87fc1055e0 0x7f87fc19acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f88017b0700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc100ee0 0x7f87fc196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8801fb1700 1 -- 172.21.15.93:0/3490169839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc100ee0 msgr2=0x7f87fc196880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8801fb1700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc100ee0 0x7f87fc196880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8801fb1700 1 -- 172.21.15.93:0/3490169839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87fc1055e0 msgr2=0x7f87fc19acf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8801fb1700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87fc1055e0 0x7f87fc19acf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8801fb1700 1 -- 172.21.15.93:0/3490169839 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87f80095f0 con 0x7f87fc100540 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8801fb1700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100540 0x7f87fc196340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00c920 tx=0x7f87ec00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f87f2ffd700 1 -- 172.21.15.93:0/3490169839 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec007b00 con 0x7f87fc100540 2023-12-14T18:25:10.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f87f2ffd700 1 -- 172.21.15.93:0/3490169839 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f87ec00ce70 con 0x7f87fc100540 2023-12-14T18:25:10.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87fc110360 con 0x7f87fc100540 2023-12-14T18:25:10.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f87f2ffd700 1 -- 172.21.15.93:0/3490169839 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87ec0054e0 con 0x7f87fc100540 2023-12-14T18:25:10.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.174+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87fc1101d0 con 0x7f87fc100540 2023-12-14T18:25:10.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.175+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87fc1104f0 con 0x7f87fc100540 2023-12-14T18:25:10.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.176+0000 7f87f2ffd700 1 -- 172.21.15.93:0/3490169839 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f87ec004750 con 0x7f87fc100540 2023-12-14T18:25:10.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.177+0000 7f87f2ffd700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87e80a1260 0x7f87e80a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:10.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.177+0000 7f88017b0700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87e80a1260 0x7f87e80a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:10.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.177+0000 7f87f2ffd700 1 -- 172.21.15.93:0/3490169839 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f87ec0cc790 con 0x7f87fc100540 2023-12-14T18:25:10.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.177+0000 7f88017b0700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87e80a1260 0x7f87e80a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87f8000c00 tx=0x7f87f8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:10.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.181+0000 7f87f2ffd700 1 -- 172.21.15.93:0/3490169839 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f87ec095b90 con 0x7f87fc100540 2023-12-14T18:25:10.370 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.369+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f87fc066ea0 con 0x7f87fc100540 2023-12-14T18:25:10.371 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.370+0000 7f87f2ffd700 1 -- 172.21.15.93:0/3490169839 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7f87ec0991b0 con 0x7f87fc100540 2023-12-14T18:25:10.372 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:10.372 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:25:10.373 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.372+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87e80a1260 msgr2=0x7f87e80a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.372+0000 7f8804215700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87e80a1260 0x7f87e80a3710 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f87f8000c00 tx=0x7f87f8005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.372+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100540 msgr2=0x7f87fc196340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.372+0000 7f8804215700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100540 0x7f87fc196340 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f87ec00c920 tx=0x7f87ec00cc30 comp rx=0 tx=0).stop 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.373+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 shutdown_connections 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.373+0000 7f8804215700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f87e80a1260 0x7f87e80a3710 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.373+0000 7f8804215700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f87fc100540 0x7f87fc196340 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.373+0000 7f8804215700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f87fc1055e0 0x7f87fc19acf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.373+0000 7f8804215700 1 --2- 172.21.15.93:0/3490169839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f87fc100ee0 0x7f87fc196880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.373+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 >> 172.21.15.93:0/3490169839 conn(0x7f87fc0fbdc0 msgr2=0x7f87fc0fc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.373+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 shutdown_connections 2023-12-14T18:25:10.374 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:10.373+0000 7f8804215700 1 -- 172.21.15.93:0/3490169839 wait complete. 2023-12-14T18:25:10.760 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2023-12-14T18:25:10.760 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd dump --format=json 2023-12-14T18:25:11.072 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:11.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:11 smithi093 ceph-mon[42070]: pgmap v241: 129 pgs: 129 active+clean; 618 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-12-14T18:25:11.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:11 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3490169839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:11.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:11 smithi097 ceph-mon[43486]: pgmap v241: 129 pgs: 129 active+clean; 618 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-12-14T18:25:11.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:11 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3490169839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:11.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:11 smithi160 ceph-mon[43840]: pgmap v241: 129 pgs: 129 active+clean; 618 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.5 KiB/s wr, 12 op/s 2023-12-14T18:25:11.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:11 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3490169839' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:12.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.115+0000 7f42fe75d700 1 -- 172.21.15.93:0/493312916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f80ffe10 msgr2=0x7f42f8103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:12.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.115+0000 7f42fe75d700 1 --2- 172.21.15.93:0/493312916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f80ffe10 0x7f42f8103de0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f42e80099c0 tx=0x7f42e8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.115+0000 7f42fe75d700 1 -- 172.21.15.93:0/493312916 shutdown_connections 2023-12-14T18:25:12.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.115+0000 7f42fe75d700 1 --2- 172.21.15.93:0/493312916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42f8104510 0x7f42f810c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.115+0000 7f42fe75d700 1 --2- 172.21.15.93:0/493312916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f80ff470 0x7f42f80ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.115+0000 7f42fe75d700 1 --2- 172.21.15.93:0/493312916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f80ffe10 0x7f42f8103de0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.115+0000 7f42fe75d700 1 -- 172.21.15.93:0/493312916 >> 172.21.15.93:0/493312916 conn(0x7f42f80770d0 msgr2=0x7f42f80774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:12.120 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.115+0000 7f42fe75d700 1 -- 172.21.15.93:0/493312916 shutdown_connections 2023-12-14T18:25:12.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.115+0000 7f42fe75d700 1 -- 172.21.15.93:0/493312916 wait complete. 2023-12-14T18:25:12.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.116+0000 7f42fe75d700 1 Processor -- start 2023-12-14T18:25:12.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.116+0000 7f42fe75d700 1 -- start start 2023-12-14T18:25:12.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.116+0000 7f42fe75d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f80ff470 0x7f42f81952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:12.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fe75d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42f80ffe10 0x7f42f8195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:12.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fe75d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f8104510 0x7f42f8199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:12.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fe75d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f819a290 con 0x7f42f8104510 2023-12-14T18:25:12.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fe75d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f819a400 con 0x7f42f80ffe10 2023-12-14T18:25:12.121 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fe75d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42f819a5c0 con 0x7f42f80ff470 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fccfa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f8104510 0x7f42f8199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fccfa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f8104510 0x7f42f8199c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59416/0 (socket says 172.21.15.93:59416) 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fccfa700 1 -- 172.21.15.93:0/2382394992 learned_addr learned my addr 172.21.15.93:0/2382394992 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42f7fff700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f80ff470 0x7f42f81952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42f77fe700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42f80ffe10 0x7f42f8195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fccfa700 1 -- 172.21.15.93:0/2382394992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f80ff470 msgr2=0x7f42f81952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fccfa700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f80ff470 0x7f42f81952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fccfa700 1 -- 172.21.15.93:0/2382394992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42f80ffe10 msgr2=0x7f42f8195810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fccfa700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42f80ffe10 0x7f42f8195810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.122 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fccfa700 1 -- 172.21.15.93:0/2382394992 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e80095f0 con 0x7f42f8104510 2023-12-14T18:25:12.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42f77fe700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42f80ffe10 0x7f42f8195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:12.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fccfa700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f8104510 0x7f42f8199c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42f000cfb0 tx=0x7f42f000e330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:12.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42f57fa700 1 -- 172.21.15.93:0/2382394992 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f0011070 con 0x7f42f8104510 2023-12-14T18:25:12.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42f810f240 con 0x7f42f8104510 2023-12-14T18:25:12.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42f57fa700 1 -- 172.21.15.93:0/2382394992 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f42f0007b30 con 0x7f42f8104510 2023-12-14T18:25:12.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42f57fa700 1 -- 172.21.15.93:0/2382394992 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f000adf0 con 0x7f42f8104510 2023-12-14T18:25:12.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.117+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42f810f0b0 con 0x7f42f8104510 2023-12-14T18:25:12.123 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.118+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42f810f3d0 con 0x7f42f8104510 2023-12-14T18:25:12.124 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.119+0000 7f42f57fa700 1 -- 172.21.15.93:0/2382394992 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f42f000e980 con 0x7f42f8104510 2023-12-14T18:25:12.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.120+0000 7f42f57fa700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f42e00a11e0 0x7f42e00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:12.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.120+0000 7f42f7fff700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f42e00a11e0 0x7f42e00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:12.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.120+0000 7f42f57fa700 1 -- 172.21.15.93:0/2382394992 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f42f00ced90 con 0x7f42f8104510 2023-12-14T18:25:12.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.120+0000 7f42f7fff700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f42e00a11e0 0x7f42e00a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42ec005950 tx=0x7f42ec00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:12.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.124+0000 7f42f57fa700 1 -- 172.21.15.93:0/2382394992 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f42f0098190 con 0x7f42f8104510 2023-12-14T18:25:12.313 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.312+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f42f8069260 con 0x7f42f8104510 2023-12-14T18:25:12.314 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.313+0000 7f42f57fa700 1 -- 172.21.15.93:0/2382394992 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v84) v1 ==== 74+0+23888 (secure 0 0 0) 0x7f42f009b7b0 con 0x7f42f8104510 2023-12-14T18:25:12.315 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:12.315 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":84,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:24:40.995454+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:25:12.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f42e00a11e0 msgr2=0x7f42e00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:12.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f42e00a11e0 0x7f42e00a3690 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f42ec005950 tx=0x7f42ec00a300 comp rx=0 tx=0).stop 2023-12-14T18:25:12.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f8104510 msgr2=0x7f42f8199c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:12.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f8104510 0x7f42f8199c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f42f000cfb0 tx=0x7f42f000e330 comp rx=0 tx=0).stop 2023-12-14T18:25:12.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 shutdown_connections 2023-12-14T18:25:12.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f42e00a11e0 0x7f42e00a3690 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.317 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f42f8104510 0x7f42f8199c80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f42f80ff470 0x7f42f81952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 --2- 172.21.15.93:0/2382394992 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f42f80ffe10 0x7f42f8195810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:12.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.315+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 >> 172.21.15.93:0/2382394992 conn(0x7f42f80770d0 msgr2=0x7f42f80fccd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:12.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.316+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 shutdown_connections 2023-12-14T18:25:12.318 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:12.316+0000 7f42fe75d700 1 -- 172.21.15.93:0/2382394992 wait complete. 2023-12-14T18:25:12.718 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.0 flush_pg_stats 2023-12-14T18:25:12.718 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.1 flush_pg_stats 2023-12-14T18:25:12.719 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.2 flush_pg_stats 2023-12-14T18:25:12.719 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.3 flush_pg_stats 2023-12-14T18:25:12.720 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.4 flush_pg_stats 2023-12-14T18:25:12.720 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.5 flush_pg_stats 2023-12-14T18:25:12.721 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.6 flush_pg_stats 2023-12-14T18:25:12.721 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.7 flush_pg_stats 2023-12-14T18:25:12.722 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.8 flush_pg_stats 2023-12-14T18:25:12.722 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.9 flush_pg_stats 2023-12-14T18:25:12.723 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.10 flush_pg_stats 2023-12-14T18:25:12.723 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph tell osd.11 flush_pg_stats 2023-12-14T18:25:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:13 smithi097 ceph-mon[43486]: pgmap v242: 129 pgs: 129 active+clean; 623 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2023-12-14T18:25:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:13 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2382394992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:13.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:13 smithi160 ceph-mon[43840]: pgmap v242: 129 pgs: 129 active+clean; 623 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2023-12-14T18:25:13.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:13 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2382394992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:13.555 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:13.571 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:13 smithi093 ceph-mon[42070]: pgmap v242: 129 pgs: 129 active+clean; 623 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 5.9 KiB/s wr, 14 op/s 2023-12-14T18:25:13.571 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:13 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2382394992' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:14.013 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:14.106 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:14.272 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:14.825 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:14.825 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:15.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.041+0000 7fa25f709700 1 -- 172.21.15.93:0/1029667832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2580ffe10 msgr2=0x7fa258103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:15.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.041+0000 7fa25f709700 1 --2- 172.21.15.93:0/1029667832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2580ffe10 0x7fa258103de0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fa24c0099c0 tx=0x7fa24c009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:15.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.041+0000 7fa25f709700 1 -- 172.21.15.93:0/1029667832 shutdown_connections 2023-12-14T18:25:15.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.041+0000 7fa25f709700 1 --2- 172.21.15.93:0/1029667832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa258104510 0x7fa25810c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:15.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.041+0000 7fa25f709700 1 --2- 172.21.15.93:0/1029667832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2580ff470 0x7fa2580ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:15.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.041+0000 7fa25f709700 1 --2- 172.21.15.93:0/1029667832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2580ffe10 0x7fa258103de0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:15.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.041+0000 7fa25f709700 1 -- 172.21.15.93:0/1029667832 >> 172.21.15.93:0/1029667832 conn(0x7fa2580770d0 msgr2=0x7fa2580774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:15.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.041+0000 7fa25f709700 1 -- 172.21.15.93:0/1029667832 shutdown_connections 2023-12-14T18:25:15.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.041+0000 7fa25f709700 1 -- 172.21.15.93:0/1029667832 wait complete. 2023-12-14T18:25:15.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25f709700 1 Processor -- start 2023-12-14T18:25:15.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25f709700 1 -- start start 2023-12-14T18:25:15.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25f709700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2580ff470 0x7fa258195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:15.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25f709700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2580ffe10 0x7fa2581958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:15.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25f709700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa258104510 0x7fa258199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:15.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25f709700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25819a320 con 0x7fa258104510 2023-12-14T18:25:15.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25f709700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25819a490 con 0x7fa2580ffe10 2023-12-14T18:25:15.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25f709700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa25819a650 con 0x7fa2580ff470 2023-12-14T18:25:15.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25dca6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa258104510 0x7fa258199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:15.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25dca6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa258104510 0x7fa258199d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59434/0 (socket says 172.21.15.93:59434) 2023-12-14T18:25:15.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25dca6700 1 -- 172.21.15.93:0/277781644 learned_addr learned my addr 172.21.15.93:0/277781644 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25cca4700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2580ffe10 0x7fa2581958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25d4a5700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2580ff470 0x7fa258195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25dca6700 1 -- 172.21.15.93:0/277781644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2580ff470 msgr2=0x7fa258195360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25dca6700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2580ff470 0x7fa258195360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25dca6700 1 -- 172.21.15.93:0/277781644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2580ffe10 msgr2=0x7fa2581958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25dca6700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2580ffe10 0x7fa2581958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25dca6700 1 -- 172.21.15.93:0/277781644 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa24c0095f0 con 0x7fa258104510 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.042+0000 7fa25d4a5700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2580ff470 0x7fa258195360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.043+0000 7fa25dca6700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa258104510 0x7fa258199d10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa25000d980 tx=0x7fa25000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.043+0000 7fa2567fc700 1 -- 172.21.15.93:0/277781644 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250004a30 con 0x7fa258104510 2023-12-14T18:25:15.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.043+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa25810f240 con 0x7fa258104510 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.043+0000 7fa2567fc700 1 -- 172.21.15.93:0/277781644 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa25000b5c0 con 0x7fa258104510 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.043+0000 7fa2567fc700 1 -- 172.21.15.93:0/277781644 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa250009b50 con 0x7fa258104510 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.043+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa25810f0b0 con 0x7fa258104510 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.044+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa2580fd6c0 con 0x7fa258104510 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.045+0000 7fa2567fc700 1 -- 172.21.15.93:0/277781644 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa250009cb0 con 0x7fa258104510 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.045+0000 7fa2567fc700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2400a11e0 0x7fa2400a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.045+0000 7fa25d4a5700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2400a11e0 0x7fa2400a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.045+0000 7fa2567fc700 1 -- 172.21.15.93:0/277781644 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa2500ccaa0 con 0x7fa258104510 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.046+0000 7fa2567fc700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] conn(0x7fa2400abd50 0x7fa2400ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:15.049 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.046+0000 7fa2567fc700 1 -- 172.21.15.93:0/277781644 --> [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa2400ae850 con 0x7fa2400abd50 2023-12-14T18:25:15.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.046+0000 7fa2567fc700 1 -- 172.21.15.93:0/277781644 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa2500d12c0 con 0x7fa258104510 2023-12-14T18:25:15.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.046+0000 7fa25d4a5700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2400a11e0 0x7fa2400a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa248009dd0 tx=0x7fa248009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:15.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.046+0000 7fa25cca4700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] conn(0x7fa2400abd50 0x7fa2400ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:15.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.046+0000 7fa25cca4700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] conn(0x7fa2400abd50 0x7fa2400ae180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:15.050 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.047+0000 7fa2567fc700 1 -- 172.21.15.93:0/277781644 <== osd.1 v2:172.21.15.97:6800/3276231390 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fa2400ae850 con 0x7fa2400abd50 2023-12-14T18:25:15.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.063+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 --> [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa25810f3d0 con 0x7fa2400abd50 2023-12-14T18:25:15.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.063+0000 7fa2567fc700 1 -- 172.21.15.93:0/277781644 <== osd.1 v2:172.21.15.97:6800/3276231390 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa25810f3d0 con 0x7fa2400abd50 2023-12-14T18:25:15.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 >> [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] conn(0x7fa2400abd50 msgr2=0x7fa2400ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:15.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] conn(0x7fa2400abd50 0x7fa2400ae180 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:15.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2400a11e0 msgr2=0x7fa2400a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:15.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2400a11e0 0x7fa2400a3690 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fa248009dd0 tx=0x7fa248009040 comp rx=0 tx=0).stop 2023-12-14T18:25:15.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa258104510 msgr2=0x7fa258199d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:15.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 --2- 172.21.15.93:0/277781644 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa258104510 0x7fa258199d10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fa25000d980 tx=0x7fa25000dc90 comp rx=0 tx=0).stop 2023-12-14T18:25:15.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25dca6700 1 -- 172.21.15.93:0/277781644 reap_dead start 2023-12-14T18:25:15.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 shutdown_connections 2023-12-14T18:25:15.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 >> 172.21.15.93:0/277781644 conn(0x7fa2580770d0 msgr2=0x7fa2580fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:15.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 shutdown_connections 2023-12-14T18:25:15.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:15.064+0000 7fa25f709700 1 -- 172.21.15.93:0/277781644 wait complete. 2023-12-14T18:25:15.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:15 smithi097 ceph-mon[43486]: pgmap v243: 129 pgs: 129 active+clean; 623 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-12-14T18:25:15.431 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:15 smithi160 ceph-mon[43840]: pgmap v243: 129 pgs: 129 active+clean; 623 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-12-14T18:25:15.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:15 smithi093 ceph-mon[42070]: pgmap v243: 129 pgs: 129 active+clean; 623 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.1 KiB/s wr, 12 op/s 2023-12-14T18:25:15.996 INFO:teuthology.orchestra.run.smithi093.stdout:55834574901 2023-12-14T18:25:15.996 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.1 2023-12-14T18:25:17.000 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:16 smithi093 ceph-mon[42070]: pgmap v244: 129 pgs: 129 active+clean; 623 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-12-14T18:25:17.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:16 smithi097 ceph-mon[43486]: pgmap v244: 129 pgs: 129 active+clean; 623 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-12-14T18:25:17.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.217+0000 7f383a6d4700 1 -- 172.21.15.93:0/2376109306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3834109ca0 msgr2=0x7f383410c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:17.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.217+0000 7f383a6d4700 1 --2- 172.21.15.93:0/2376109306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3834109ca0 0x7f383410c080 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f382c00b0a0 tx=0x7f382c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:17.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.217+0000 7f383a6d4700 1 -- 172.21.15.93:0/2376109306 shutdown_connections 2023-12-14T18:25:17.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.217+0000 7f383a6d4700 1 --2- 172.21.15.93:0/2376109306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3834109ca0 0x7f383410c080 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:17.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.217+0000 7f383a6d4700 1 --2- 172.21.15.93:0/2376109306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38341006f0 0x7f3834100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:17.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.217+0000 7f383a6d4700 1 --2- 172.21.15.93:0/2376109306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3834101000 0x7f3834109760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:17.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.217+0000 7f383a6d4700 1 -- 172.21.15.93:0/2376109306 >> 172.21.15.93:0/2376109306 conn(0x7f3834074ba0 msgr2=0x7f3834074fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:17.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.217+0000 7f383a6d4700 1 -- 172.21.15.93:0/2376109306 shutdown_connections 2023-12-14T18:25:17.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.218+0000 7f383a6d4700 1 -- 172.21.15.93:0/2376109306 wait complete. 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.218+0000 7f383a6d4700 1 Processor -- start 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.218+0000 7f383a6d4700 1 -- start start 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.218+0000 7f383a6d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38341006f0 0x7f3834196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.218+0000 7f383a6d4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3834101000 0x7f3834196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.218+0000 7f383a6d4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3834109ca0 0x7f383419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f383a6d4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383419b400 con 0x7f3834109ca0 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f383a6d4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383419b570 con 0x7f3834101000 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f383a6d4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383419b730 con 0x7f38341006f0 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3838c71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3834109ca0 0x7f383419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3838c71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3834109ca0 0x7f383419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59444/0 (socket says 172.21.15.93:59444) 2023-12-14T18:25:17.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3838c71700 1 -- 172.21.15.93:0/851835831 learned_addr learned my addr 172.21.15.93:0/851835831 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:17.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f38337fe700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3834101000 0x7f3834196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:17.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3833fff700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38341006f0 0x7f3834196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:17.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3838c71700 1 -- 172.21.15.93:0/851835831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38341006f0 msgr2=0x7f3834196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:17.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3838c71700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f38341006f0 0x7f3834196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:17.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3838c71700 1 -- 172.21.15.93:0/851835831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3834101000 msgr2=0x7f3834196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:17.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3838c71700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3834101000 0x7f3834196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:17.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3838c71700 1 -- 172.21.15.93:0/851835831 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f382c009d00 con 0x7f3834109ca0 2023-12-14T18:25:17.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f3838c71700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3834109ca0 0x7f383419adf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f382c0062a0 tx=0x7f382c0079b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:17.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f38317fa700 1 -- 172.21.15.93:0/851835831 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c00e040 con 0x7f3834109ca0 2023-12-14T18:25:17.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3834110490 con 0x7f3834109ca0 2023-12-14T18:25:17.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f38317fa700 1 -- 172.21.15.93:0/851835831 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f382c004ce0 con 0x7f3834109ca0 2023-12-14T18:25:17.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f38317fa700 1 -- 172.21.15.93:0/851835831 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c01d480 con 0x7f3834109ca0 2023-12-14T18:25:17.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.219+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3834110300 con 0x7f3834109ca0 2023-12-14T18:25:17.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.220+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f3834061270 con 0x7f3834109ca0 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.222+0000 7f38317fa700 1 -- 172.21.15.93:0/851835831 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f382c004000 con 0x7f3834109ca0 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.222+0000 7f38317fa700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f381c0a11e0 0x7f381c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.222+0000 7f38317fa700 1 -- 172.21.15.93:0/851835831 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f382c0ced70 con 0x7f3834109ca0 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.222+0000 7f3833fff700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f381c0a11e0 0x7f381c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.222+0000 7f38317fa700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] conn(0x7f381c0abc20 0x7f381c0ae050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.222+0000 7f38317fa700 1 -- 172.21.15.93:0/851835831 --> [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f381c0ae720 con 0x7f381c0abc20 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.222+0000 7f38317fa700 1 -- 172.21.15.93:0/851835831 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f382c0cf180 con 0x7f3834109ca0 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.223+0000 7f3833fff700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f381c0a11e0 0x7f381c0a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3824005950 tx=0x7f382400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.223+0000 7f38337fe700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] conn(0x7f381c0abc20 0x7f381c0ae050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:17.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.223+0000 7f38337fe700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] conn(0x7f381c0abc20 0x7f381c0ae050 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:17.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.224+0000 7f38317fa700 1 -- 172.21.15.93:0/851835831 <== osd.4 v2:172.21.15.97:6808/3095116912 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f381c0ae720 con 0x7f381c0abc20 2023-12-14T18:25:17.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.240+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 --> [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f3834110620 con 0x7f381c0abc20 2023-12-14T18:25:17.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.240+0000 7f38317fa700 1 -- 172.21.15.93:0/851835831 <== osd.4 v2:172.21.15.97:6808/3095116912 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f3834110620 con 0x7f381c0abc20 2023-12-14T18:25:17.242 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 >> [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] conn(0x7f381c0abc20 msgr2=0x7f381c0ae050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] conn(0x7f381c0abc20 0x7f381c0ae050 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f381c0a11e0 msgr2=0x7f381c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f381c0a11e0 0x7f381c0a3690 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f3824005950 tx=0x7f382400b410 comp rx=0 tx=0).stop 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3834109ca0 msgr2=0x7f383419adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 --2- 172.21.15.93:0/851835831 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3834109ca0 0x7f383419adf0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f382c0062a0 tx=0x7f382c0079b0 comp rx=0 tx=0).stop 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f3838c71700 1 -- 172.21.15.93:0/851835831 reap_dead start 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 shutdown_connections 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 >> 172.21.15.93:0/851835831 conn(0x7f3834074ba0 msgr2=0x7f3834101700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 shutdown_connections 2023-12-14T18:25:17.243 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:17.241+0000 7f383a6d4700 1 -- 172.21.15.93:0/851835831 wait complete. 2023-12-14T18:25:17.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:16 smithi160 ceph-mon[43840]: pgmap v244: 129 pgs: 129 active+clean; 623 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-12-14T18:25:17.997 INFO:teuthology.orchestra.run.smithi093.stdout:133143986219 2023-12-14T18:25:17.997 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.4 2023-12-14T18:25:18.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.038+0000 7f3532b71700 1 -- 172.21.15.93:0/1311774231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100540 msgr2=0x7f352c100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.038+0000 7f3532b71700 1 --2- 172.21.15.93:0/1311774231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100540 0x7f352c100910 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f35200099c0 tx=0x7f3520009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.038+0000 7f3532b71700 1 -- 172.21.15.93:0/1311774231 shutdown_connections 2023-12-14T18:25:18.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.038+0000 7f3532b71700 1 --2- 172.21.15.93:0/1311774231 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100540 0x7f352c100910 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.039 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.038+0000 7f3532b71700 1 --2- 172.21.15.93:0/1311774231 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f352c1055e0 0x7f352c10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.038+0000 7f3532b71700 1 --2- 172.21.15.93:0/1311774231 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352c100ee0 0x7f352c104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.038+0000 7f3532b71700 1 -- 172.21.15.93:0/1311774231 >> 172.21.15.93:0/1311774231 conn(0x7f352c0fbdc0 msgr2=0x7f352c0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:18.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.038+0000 7f3532b71700 1 -- 172.21.15.93:0/1311774231 shutdown_connections 2023-12-14T18:25:18.040 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.038+0000 7f3532b71700 1 -- 172.21.15.93:0/1311774231 wait complete. 2023-12-14T18:25:18.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f3532b71700 1 Processor -- start 2023-12-14T18:25:18.041 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f3532b71700 1 -- start start 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f3532b71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100540 0x7f352c06b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f3532b71700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f352c100ee0 0x7f352c06b540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f3532b71700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352c1055e0 0x7f352c199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f3532b71700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c199f00 con 0x7f352c100540 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f3532b71700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c19a070 con 0x7f352c1055e0 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f3532b71700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352c19a230 con 0x7f352c100ee0 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f352bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f352c100ee0 0x7f352c06b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f352bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f352c100ee0 0x7f352c06b540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:42622/0 (socket says 172.21.15.93:42622) 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f352bfff700 1 -- 172.21.15.93:0/1420451008 learned_addr learned my addr 172.21.15.93:0/1420451008 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:18.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f353110e700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352c1055e0 0x7f352c199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.039+0000 7f353090d700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100540 0x7f352c06b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f353090d700 1 -- 172.21.15.93:0/1420451008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f352c100ee0 msgr2=0x7f352c06b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f353090d700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f352c100ee0 0x7f352c06b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f353090d700 1 -- 172.21.15.93:0/1420451008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352c1055e0 msgr2=0x7f352c199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f353090d700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f352c1055e0 0x7f352c199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f353090d700 1 -- 172.21.15.93:0/1420451008 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35200095f0 con 0x7f352c100540 2023-12-14T18:25:18.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f353090d700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100540 0x7f352c06b000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3520009990 tx=0x7f3520004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f3529ffb700 1 -- 172.21.15.93:0/1420451008 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f352001d070 con 0x7f352c100540 2023-12-14T18:25:18.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f3532b71700 1 -- 172.21.15.93:0/1420451008 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f352c110320 con 0x7f352c100540 2023-12-14T18:25:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f3529ffb700 1 -- 172.21.15.93:0/1420451008 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3520003f90 con 0x7f352c100540 2023-12-14T18:25:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.040+0000 7f3529ffb700 1 -- 172.21.15.93:0/1420451008 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3520022b50 con 0x7f352c100540 2023-12-14T18:25:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.041+0000 7f3532b71700 1 -- 172.21.15.93:0/1420451008 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f352c110190 con 0x7f352c100540 2023-12-14T18:25:18.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.041+0000 7f35177fe700 1 -- 172.21.15.93:0/1420451008 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f350c000fc0 con 0x7f352c100540 2023-12-14T18:25:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.044+0000 7f3529ffb700 1 -- 172.21.15.93:0/1420451008 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3520003ad0 con 0x7f352c100540 2023-12-14T18:25:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.045+0000 7f3529ffb700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f351c0a11e0 0x7f351c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.046 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.045+0000 7f3529ffb700 1 -- 172.21.15.93:0/1420451008 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f35200cea60 con 0x7f352c100540 2023-12-14T18:25:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.045+0000 7f352bfff700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f351c0a11e0 0x7f351c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.045+0000 7f3529ffb700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] conn(0x7f351c0abd50 0x7f351c0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.045+0000 7f3529ffb700 1 -- 172.21.15.93:0/1420451008 --> [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f351c0ae850 con 0x7f351c0abd50 2023-12-14T18:25:18.047 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.045+0000 7f3529ffb700 1 -- 172.21.15.93:0/1420451008 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f35200cee70 con 0x7f352c100540 2023-12-14T18:25:18.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.046+0000 7f352bfff700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f351c0a11e0 0x7f351c0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f352c06c620 tx=0x7f351800b480 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.046+0000 7f353110e700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] conn(0x7f351c0abd50 0x7f351c0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.046+0000 7f353110e700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] conn(0x7f351c0abd50 0x7f351c0ae180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.048 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.047+0000 7f3529ffb700 1 -- 172.21.15.93:0/1420451008 <== osd.7 v2:172.21.15.97:6816/1426225102 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f351c0ae850 con 0x7f351c0abd50 2023-12-14T18:25:18.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.072+0000 7f35177fe700 1 -- 172.21.15.93:0/1420451008 --> [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f350c002d40 con 0x7f351c0abd50 2023-12-14T18:25:18.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f3529ffb700 1 -- 172.21.15.93:0/1420451008 <== osd.7 v2:172.21.15.97:6816/1426225102 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f350c002d40 con 0x7f351c0abd50 2023-12-14T18:25:18.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 -- 172.21.15.93:0/1420451008 >> [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] conn(0x7f351c0abd50 msgr2=0x7f351c0ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] conn(0x7f351c0abd50 0x7f351c0ae180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 -- 172.21.15.93:0/1420451008 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f351c0a11e0 msgr2=0x7f351c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f351c0a11e0 0x7f351c0a3690 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f352c06c620 tx=0x7f351800b480 comp rx=0 tx=0).stop 2023-12-14T18:25:18.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 -- 172.21.15.93:0/1420451008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100540 msgr2=0x7f352c06b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 --2- 172.21.15.93:0/1420451008 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f352c100540 0x7f352c06b000 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f3520009990 tx=0x7f3520004730 comp rx=0 tx=0).stop 2023-12-14T18:25:18.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f353110e700 1 -- 172.21.15.93:0/1420451008 reap_dead start 2023-12-14T18:25:18.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 -- 172.21.15.93:0/1420451008 shutdown_connections 2023-12-14T18:25:18.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 -- 172.21.15.93:0/1420451008 >> 172.21.15.93:0/1420451008 conn(0x7f352c0fbdc0 msgr2=0x7f352c0fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:18.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 -- 172.21.15.93:0/1420451008 shutdown_connections 2023-12-14T18:25:18.076 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.073+0000 7f35177fe700 1 -- 172.21.15.93:0/1420451008 wait complete. 2023-12-14T18:25:18.092 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:18.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.129+0000 7f83c14d3700 1 -- 172.21.15.93:0/539441801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83bc104500 msgr2=0x7f83bc10c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.129+0000 7f83c14d3700 1 --2- 172.21.15.93:0/539441801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83bc104500 0x7f83bc10c080 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f83b400b0d0 tx=0x7f83b400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.130+0000 7f83c14d3700 1 -- 172.21.15.93:0/539441801 shutdown_connections 2023-12-14T18:25:18.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.130+0000 7f83c14d3700 1 --2- 172.21.15.93:0/539441801 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83bc104500 0x7f83bc10c080 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.130+0000 7f83c14d3700 1 --2- 172.21.15.93:0/539441801 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83bc0ff460 0x7f83bc0ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.130+0000 7f83c14d3700 1 --2- 172.21.15.93:0/539441801 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83bc0ffe00 0x7f83bc103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.130+0000 7f83c14d3700 1 -- 172.21.15.93:0/539441801 >> 172.21.15.93:0/539441801 conn(0x7f83bc0770a0 msgr2=0x7f83bc0774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:18.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.130+0000 7f83c14d3700 1 -- 172.21.15.93:0/539441801 shutdown_connections 2023-12-14T18:25:18.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.130+0000 7f83c14d3700 1 -- 172.21.15.93:0/539441801 wait complete. 2023-12-14T18:25:18.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.130+0000 7f83c14d3700 1 Processor -- start 2023-12-14T18:25:18.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.130+0000 7f83c14d3700 1 -- start start 2023-12-14T18:25:18.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83c14d3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83bc0ff460 0x7f83bc195140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83c14d3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83c14d3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83bc104500 0x7f83bc195bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83c14d3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1a4980 con 0x7f83bc0ffe00 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83c14d3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1a4af0 con 0x7f83bc0ff460 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83c14d3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83bc1a4cb0 con 0x7f83bc104500 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83ba7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83ba7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59482/0 (socket says 172.21.15.93:59482) 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83ba7fc700 1 -- 172.21.15.93:0/2302204242 learned_addr learned my addr 172.21.15.93:0/2302204242 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83bb7fe700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83bc104500 0x7f83bc195bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83baffd700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83bc0ff460 0x7f83bc195140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83ba7fc700 1 -- 172.21.15.93:0/2302204242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83bc104500 msgr2=0x7f83bc195bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83ba7fc700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f83bc104500 0x7f83bc195bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83ba7fc700 1 -- 172.21.15.93:0/2302204242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83bc0ff460 msgr2=0x7f83bc195140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83ba7fc700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f83bc0ff460 0x7f83bc195140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83ba7fc700 1 -- 172.21.15.93:0/2302204242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83b4009d00 con 0x7f83bc0ffe00 2023-12-14T18:25:18.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.131+0000 7f83ba7fc700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f83b000d980 tx=0x7f83b000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.132+0000 7f83a3fff700 1 -- 172.21.15.93:0/2302204242 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b0009b40 con 0x7f83bc0ffe00 2023-12-14T18:25:18.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.132+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83bc10f240 con 0x7f83bc0ffe00 2023-12-14T18:25:18.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.132+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83bc10f0b0 con 0x7f83bc0ffe00 2023-12-14T18:25:18.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.132+0000 7f83a3fff700 1 -- 172.21.15.93:0/2302204242 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f83b000de60 con 0x7f83bc0ffe00 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.132+0000 7f83a3fff700 1 -- 172.21.15.93:0/2302204242 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83b000f620 con 0x7f83bc0ffe00 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.133+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f83a8000ff0 con 0x7f83bc0ffe00 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.134+0000 7f83a3fff700 1 -- 172.21.15.93:0/2302204242 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f83b0009ca0 con 0x7f83bc0ffe00 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.135+0000 7f83a3fff700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f83a40a12e0 0x7f83a40a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.135+0000 7f83a3fff700 1 -- 172.21.15.93:0/2302204242 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f83b00cca60 con 0x7f83bc0ffe00 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.135+0000 7f83baffd700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f83a40a12e0 0x7f83a40a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.135+0000 7f83a3fff700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] conn(0x7f83a40abdd0 0x7f83a40ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.135+0000 7f83a3fff700 1 -- 172.21.15.93:0/2302204242 --> [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f83a40ae8d0 con 0x7f83a40abdd0 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.135+0000 7f83a3fff700 1 -- 172.21.15.93:0/2302204242 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f83b0095e00 con 0x7f83bc0ffe00 2023-12-14T18:25:18.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.135+0000 7f83baffd700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f83a40a12e0 0x7f83a40a3790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83ac005fd0 tx=0x7f83ac005f60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.135+0000 7f83bb7fe700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] conn(0x7f83a40abdd0 0x7f83a40ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.136+0000 7f83bb7fe700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] conn(0x7f83a40abdd0 0x7f83a40ae200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.137+0000 7f83a3fff700 1 -- 172.21.15.93:0/2302204242 <== osd.8 v2:172.21.15.160:6816/3984975827 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f83a40ae8d0 con 0x7f83a40abdd0 2023-12-14T18:25:18.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.153+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 --> [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f83a8002d70 con 0x7f83a40abdd0 2023-12-14T18:25:18.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83a3fff700 1 -- 172.21.15.93:0/2302204242 <== osd.8 v2:172.21.15.160:6816/3984975827 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f83a8002d70 con 0x7f83a40abdd0 2023-12-14T18:25:18.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 >> [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] conn(0x7f83a40abdd0 msgr2=0x7f83a40ae200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83c14d3700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] conn(0x7f83a40abdd0 0x7f83a40ae200 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f83a40a12e0 msgr2=0x7f83a40a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83c14d3700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f83a40a12e0 0x7f83a40a3790 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f83ac005fd0 tx=0x7f83ac005f60 comp rx=0 tx=0).stop 2023-12-14T18:25:18.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83bc0ffe00 msgr2=0x7f83bc195680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83c14d3700 1 --2- 172.21.15.93:0/2302204242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f83bc0ffe00 0x7f83bc195680 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f83b000d980 tx=0x7f83b000dd40 comp rx=0 tx=0).stop 2023-12-14T18:25:18.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83bb7fe700 1 -- 172.21.15.93:0/2302204242 reap_dead start 2023-12-14T18:25:18.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 shutdown_connections 2023-12-14T18:25:18.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.154+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 >> 172.21.15.93:0/2302204242 conn(0x7f83bc0770a0 msgr2=0x7f83bc0fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:18.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.155+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 shutdown_connections 2023-12-14T18:25:18.157 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.155+0000 7f83c14d3700 1 -- 172.21.15.93:0/2302204242 wait complete. 2023-12-14T18:25:18.699 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:18.700 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:18.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.842+0000 7efeb79db700 1 -- 172.21.15.93:0/3170619475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00ff470 msgr2=0x7efeb00ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.842+0000 7efeb79db700 1 --2- 172.21.15.93:0/3170619475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00ff470 0x7efeb00ff840 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7efea00099c0 tx=0x7efea0009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.843+0000 7efeb79db700 1 -- 172.21.15.93:0/3170619475 shutdown_connections 2023-12-14T18:25:18.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.843+0000 7efeb79db700 1 --2- 172.21.15.93:0/3170619475 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00ff470 0x7efeb00ff840 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.843+0000 7efeb79db700 1 --2- 172.21.15.93:0/3170619475 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efeb0104510 0x7efeb010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.843+0000 7efeb79db700 1 --2- 172.21.15.93:0/3170619475 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb00ffe10 0x7efeb0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.843+0000 7efeb79db700 1 -- 172.21.15.93:0/3170619475 >> 172.21.15.93:0/3170619475 conn(0x7efeb00770d0 msgr2=0x7efeb00774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:18.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.843+0000 7efeb79db700 1 -- 172.21.15.93:0/3170619475 shutdown_connections 2023-12-14T18:25:18.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.843+0000 7efeb79db700 1 -- 172.21.15.93:0/3170619475 wait complete. 2023-12-14T18:25:18.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.843+0000 7efeb79db700 1 Processor -- start 2023-12-14T18:25:18.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.843+0000 7efeb79db700 1 -- start start 2023-12-14T18:25:18.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb79db700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efeb00ff470 0x7efeb01952b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb79db700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00ffe10 0x7efeb01957f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb79db700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb0104510 0x7efeb0199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb79db700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb019a270 con 0x7efeb00ffe10 2023-12-14T18:25:18.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb79db700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb019a3e0 con 0x7efeb00ff470 2023-12-14T18:25:18.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb79db700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efeb019a5a0 con 0x7efeb0104510 2023-12-14T18:25:18.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb4f76700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00ffe10 0x7efeb01957f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb4f76700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00ffe10 0x7efeb01957f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59502/0 (socket says 172.21.15.93:59502) 2023-12-14T18:25:18.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb4f76700 1 -- 172.21.15.93:0/1451666402 learned_addr learned my addr 172.21.15.93:0/1451666402 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:18.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb5777700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efeb00ff470 0x7efeb01952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb5f78700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb0104510 0x7efeb0199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb4f76700 1 -- 172.21.15.93:0/1451666402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb0104510 msgr2=0x7efeb0199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb4f76700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb0104510 0x7efeb0199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb4f76700 1 -- 172.21.15.93:0/1451666402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efeb00ff470 msgr2=0x7efeb01952b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb4f76700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efeb00ff470 0x7efeb01952b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb4f76700 1 -- 172.21.15.93:0/1451666402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00095f0 con 0x7efeb00ffe10 2023-12-14T18:25:18.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb5f78700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efeb0104510 0x7efeb0199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:18.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb5777700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efeb00ff470 0x7efeb01952b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:25:18.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.844+0000 7efeb4f76700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00ffe10 0x7efeb01957f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efeac00c8f0 tx=0x7efeac00cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.845+0000 7efea67fc700 1 -- 172.21.15.93:0/1451666402 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac007940 con 0x7efeb00ffe10 2023-12-14T18:25:18.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.845+0000 7efea67fc700 1 -- 172.21.15.93:0/1451666402 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efeac00f450 con 0x7efeb00ffe10 2023-12-14T18:25:18.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.845+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efeb010f250 con 0x7efeb00ffe10 2023-12-14T18:25:18.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.845+0000 7efea67fc700 1 -- 172.21.15.93:0/1451666402 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeac018660 con 0x7efeb00ffe10 2023-12-14T18:25:18.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.845+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efeb010f0c0 con 0x7efeb00ffe10 2023-12-14T18:25:18.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.846+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efeb0103810 con 0x7efeb00ffe10 2023-12-14T18:25:18.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.847+0000 7efea67fc700 1 -- 172.21.15.93:0/1451666402 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efeac00f5c0 con 0x7efeb00ffe10 2023-12-14T18:25:18.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.847+0000 7efea67fc700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efe9c0a11e0 0x7efe9c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.848+0000 7efeb5777700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efe9c0a11e0 0x7efe9c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.848+0000 7efea67fc700 1 -- 172.21.15.93:0/1451666402 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efeac0cce70 con 0x7efeb00ffe10 2023-12-14T18:25:18.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.848+0000 7efea67fc700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] conn(0x7efe9c0abd50 0x7efe9c0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:18.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.848+0000 7efea67fc700 1 -- 172.21.15.93:0/1451666402 --> [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efe9c0ae850 con 0x7efe9c0abd50 2023-12-14T18:25:18.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.848+0000 7efea67fc700 1 -- 172.21.15.93:0/1451666402 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7efeac0cd280 con 0x7efeb00ffe10 2023-12-14T18:25:18.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.848+0000 7efeb5f78700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] conn(0x7efe9c0abd50 0x7efe9c0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:18.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.848+0000 7efeb5777700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efe9c0a11e0 0x7efe9c0a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efea0000c00 tx=0x7efea0003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.848+0000 7efeb5f78700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] conn(0x7efe9c0abd50 0x7efe9c0ae180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:18.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.849+0000 7efea67fc700 1 -- 172.21.15.93:0/1451666402 <== osd.9 v2:172.21.15.93:6826/4200476017 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7efe9c0ae850 con 0x7efe9c0abd50 2023-12-14T18:25:18.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 --> [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7efeb010f3e0 con 0x7efe9c0abd50 2023-12-14T18:25:18.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efea67fc700 1 -- 172.21.15.93:0/1451666402 <== osd.9 v2:172.21.15.93:6826/4200476017 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7efeb010f3e0 con 0x7efe9c0abd50 2023-12-14T18:25:18.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] conn(0x7efe9c0abd50 msgr2=0x7efe9c0ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efeb79db700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] conn(0x7efe9c0abd50 0x7efe9c0ae180 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:18.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efe9c0a11e0 msgr2=0x7efe9c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efeb79db700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efe9c0a11e0 0x7efe9c0a3690 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efea0000c00 tx=0x7efea0003680 comp rx=0 tx=0).stop 2023-12-14T18:25:18.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00ffe10 msgr2=0x7efeb01957f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:18.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efeb79db700 1 --2- 172.21.15.93:0/1451666402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efeb00ffe10 0x7efeb01957f0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7efeac00c8f0 tx=0x7efeac00cc00 comp rx=0 tx=0).stop 2023-12-14T18:25:18.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efeb5f78700 1 -- 172.21.15.93:0/1451666402 reap_dead start 2023-12-14T18:25:18.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.865+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 shutdown_connections 2023-12-14T18:25:18.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.866+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 >> 172.21.15.93:0/1451666402 conn(0x7efeb00770d0 msgr2=0x7efeb0102dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:18.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.866+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 shutdown_connections 2023-12-14T18:25:18.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:18.866+0000 7efeb79db700 1 -- 172.21.15.93:0/1451666402 wait complete. 2023-12-14T18:25:18.949 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:18.955 INFO:teuthology.orchestra.run.smithi093.stdout:236223201309 2023-12-14T18:25:18.955 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.8 2023-12-14T18:25:19.200 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:19.299 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:19 smithi093 ceph-mon[42070]: pgmap v245: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-12-14T18:25:19.319 INFO:teuthology.orchestra.run.smithi093.stdout:206158430240 2023-12-14T18:25:19.319 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.7 2023-12-14T18:25:19.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:19 smithi097 ceph-mon[43486]: pgmap v245: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-12-14T18:25:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:19 smithi160 ceph-mon[43840]: pgmap v245: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 8.2 KiB/s rd, 5.0 KiB/s wr, 9 op/s 2023-12-14T18:25:19.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.985+0000 7f760bac5700 1 -- 172.21.15.93:0/1518349445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76040fe830 msgr2=0x7f76040feca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:19.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.985+0000 7f760bac5700 1 --2- 172.21.15.93:0/1518349445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76040fe830 0x7f76040feca0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f75f4009990 tx=0x7f75f4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:19.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.986+0000 7f760bac5700 1 -- 172.21.15.93:0/1518349445 shutdown_connections 2023-12-14T18:25:19.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.986+0000 7f760bac5700 1 --2- 172.21.15.93:0/1518349445 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76040ff1e0 0x7f760410d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:19.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.986+0000 7f760bac5700 1 --2- 172.21.15.93:0/1518349445 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7604105290 0x7f7604105660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:19.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.986+0000 7f760bac5700 1 --2- 172.21.15.93:0/1518349445 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76040fe830 0x7f76040feca0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:19.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.986+0000 7f760bac5700 1 -- 172.21.15.93:0/1518349445 >> 172.21.15.93:0/1518349445 conn(0x7f76040758a0 msgr2=0x7f7604075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.986+0000 7f760bac5700 1 -- 172.21.15.93:0/1518349445 shutdown_connections 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.986+0000 7f760bac5700 1 -- 172.21.15.93:0/1518349445 wait complete. 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.986+0000 7f760bac5700 1 Processor -- start 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f760bac5700 1 -- start start 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f760bac5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76040fe830 0x7f7604196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f760bac5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76040ff1e0 0x7f76041968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f760bac5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7604105290 0x7f760419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f760bac5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760419b340 con 0x7f76040ff1e0 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f760bac5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760419b4b0 con 0x7f76040fe830 2023-12-14T18:25:19.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f760bac5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760419b670 con 0x7f7604105290 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f7609060700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76040ff1e0 0x7f76041968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f7609060700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76040ff1e0 0x7f76041968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59528/0 (socket says 172.21.15.93:59528) 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f7609060700 1 -- 172.21.15.93:0/33072047 learned_addr learned my addr 172.21.15.93:0/33072047 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f7609861700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76040fe830 0x7f7604196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f760a062700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7604105290 0x7f760419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f7609060700 1 -- 172.21.15.93:0/33072047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7604105290 msgr2=0x7f760419ad30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f7609060700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7604105290 0x7f760419ad30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f7609060700 1 -- 172.21.15.93:0/33072047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76040fe830 msgr2=0x7f7604196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f7609060700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f76040fe830 0x7f7604196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:19.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f7609060700 1 -- 172.21.15.93:0/33072047 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75f40095f0 con 0x7f76040ff1e0 2023-12-14T18:25:19.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.987+0000 7f760a062700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7604105290 0x7f760419ad30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:19.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.988+0000 7f7609060700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76040ff1e0 0x7f76041968c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f75f40095c0 tx=0x7f75f400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:19.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.988+0000 7f75faffd700 1 -- 172.21.15.93:0/33072047 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f401d070 con 0x7f76040ff1e0 2023-12-14T18:25:19.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.988+0000 7f75faffd700 1 -- 172.21.15.93:0/33072047 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f75f40054f0 con 0x7f76040ff1e0 2023-12-14T18:25:19.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.988+0000 7f75faffd700 1 -- 172.21.15.93:0/33072047 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75f4004b30 con 0x7f76040ff1e0 2023-12-14T18:25:19.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.988+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7604110460 con 0x7f76040ff1e0 2023-12-14T18:25:19.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.988+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76041102d0 con 0x7f76040ff1e0 2023-12-14T18:25:19.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.989+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f76040fd9f0 con 0x7f76040ff1e0 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.990+0000 7f75faffd700 1 -- 172.21.15.93:0/33072047 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f75f400fb80 con 0x7f76040ff1e0 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.991+0000 7f75faffd700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f75f00a11e0 0x7f75f00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.991+0000 7f7609861700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f75f00a11e0 0x7f75f00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.991+0000 7f75faffd700 1 -- 172.21.15.93:0/33072047 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f75f40ce3d0 con 0x7f76040ff1e0 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.991+0000 7f75faffd700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] conn(0x7f75f00abd50 0x7f75f00ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.991+0000 7f75faffd700 1 -- 172.21.15.93:0/33072047 --> [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f75f00ae850 con 0x7f75f00abd50 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.991+0000 7f75faffd700 1 -- 172.21.15.93:0/33072047 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f75f40ce740 con 0x7f76040ff1e0 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.991+0000 7f7609861700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f75f00a11e0 0x7f75f00a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7600005950 tx=0x7f760000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.991+0000 7f760a062700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] conn(0x7f75f00abd50 0x7f75f00ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:19.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.992+0000 7f760a062700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] conn(0x7f75f00abd50 0x7f75f00ae180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:19.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:19.993+0000 7f75faffd700 1 -- 172.21.15.93:0/33072047 <== osd.11 v2:172.21.15.160:6824/747472238 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f75f00ae850 con 0x7f75f00abd50 2023-12-14T18:25:20.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.008+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 --> [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f760419bbb0 con 0x7f75f00abd50 2023-12-14T18:25:20.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.009+0000 7f75faffd700 1 -- 172.21.15.93:0/33072047 <== osd.11 v2:172.21.15.160:6824/747472238 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f760419bbb0 con 0x7f75f00abd50 2023-12-14T18:25:20.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 >> [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] conn(0x7f75f00abd50 msgr2=0x7f75f00ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:20.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] conn(0x7f75f00abd50 0x7f75f00ae180 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:20.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f75f00a11e0 msgr2=0x7f75f00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:20.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f75f00a11e0 0x7f75f00a3690 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f7600005950 tx=0x7f760000b410 comp rx=0 tx=0).stop 2023-12-14T18:25:20.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76040ff1e0 msgr2=0x7f76041968c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:20.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 --2- 172.21.15.93:0/33072047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76040ff1e0 0x7f76041968c0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f75f40095c0 tx=0x7f75f400f690 comp rx=0 tx=0).stop 2023-12-14T18:25:20.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760a062700 1 -- 172.21.15.93:0/33072047 reap_dead start 2023-12-14T18:25:20.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 shutdown_connections 2023-12-14T18:25:20.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 >> 172.21.15.93:0/33072047 conn(0x7f76040758a0 msgr2=0x7f76040fd020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:20.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 shutdown_connections 2023-12-14T18:25:20.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.010+0000 7f760bac5700 1 -- 172.21.15.93:0/33072047 wait complete. 2023-12-14T18:25:20.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.048+0000 7f294789c700 1 -- 172.21.15.93:0/1301491472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29400fe9a0 msgr2=0x7f29400fee10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:20.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.048+0000 7f294789c700 1 --2- 172.21.15.93:0/1301491472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29400fe9a0 0x7f29400fee10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f2930009990 tx=0x7f2930009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:20.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.049+0000 7f294789c700 1 -- 172.21.15.93:0/1301491472 shutdown_connections 2023-12-14T18:25:20.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.049+0000 7f294789c700 1 --2- 172.21.15.93:0/1301491472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29400ff350 0x7f294010b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:20.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.049+0000 7f294789c700 1 --2- 172.21.15.93:0/1301491472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29401049a0 0x7f2940104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:20.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.049+0000 7f294789c700 1 --2- 172.21.15.93:0/1301491472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29400fe9a0 0x7f29400fee10 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:20.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.049+0000 7f294789c700 1 -- 172.21.15.93:0/1301491472 >> 172.21.15.93:0/1301491472 conn(0x7f29400758a0 msgr2=0x7f2940075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:20.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.049+0000 7f294789c700 1 -- 172.21.15.93:0/1301491472 shutdown_connections 2023-12-14T18:25:20.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.049+0000 7f294789c700 1 -- 172.21.15.93:0/1301491472 wait complete. 2023-12-14T18:25:20.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.049+0000 7f294789c700 1 Processor -- start 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.050+0000 7f294789c700 1 -- start start 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.050+0000 7f294789c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29400fe9a0 0x7f2940198dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.050+0000 7f294789c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29400ff350 0x7f2940199300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.050+0000 7f294789c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29401049a0 0x7f294019ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.050+0000 7f294789c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294019d4a0 con 0x7f29400ff350 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.050+0000 7f294789c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294019d610 con 0x7f29401049a0 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.050+0000 7f294789c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f294019d7d0 con 0x7f29400fe9a0 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945e39700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29401049a0 0x7f294019ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945e39700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29401049a0 0x7f294019ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:43588/0 (socket says 172.21.15.93:43588) 2023-12-14T18:25:20.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945e39700 1 -- 172.21.15.93:0/3423510425 learned_addr learned my addr 172.21.15.93:0/3423510425 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:20.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2944e37700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29400ff350 0x7f2940199300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:20.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945638700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29400fe9a0 0x7f2940198dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:20.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945e39700 1 -- 172.21.15.93:0/3423510425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29400fe9a0 msgr2=0x7f2940198dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:20.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945e39700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f29400fe9a0 0x7f2940198dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:20.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945e39700 1 -- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29400ff350 msgr2=0x7f2940199300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:20.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945e39700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29400ff350 0x7f2940199300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:20.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945e39700 1 -- 172.21.15.93:0/3423510425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29300095f0 con 0x7f29401049a0 2023-12-14T18:25:20.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2944e37700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f29400ff350 0x7f2940199300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:20.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f2945e39700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29401049a0 0x7f294019ce90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2938007ee0 tx=0x7f293800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:20.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f29367fc700 1 -- 172.21.15.93:0/3423510425 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938017070 con 0x7f29401049a0 2023-12-14T18:25:20.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f29367fc700 1 -- 172.21.15.93:0/3423510425 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f293800f040 con 0x7f29401049a0 2023-12-14T18:25:20.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.051+0000 7f29367fc700 1 -- 172.21.15.93:0/3423510425 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2938005740 con 0x7f29401049a0 2023-12-14T18:25:20.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.052+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f294010f220 con 0x7f29401049a0 2023-12-14T18:25:20.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.052+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294010f090 con 0x7f29401049a0 2023-12-14T18:25:20.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.053+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f29400fd5e0 con 0x7f29401049a0 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.055+0000 7f29367fc700 1 -- 172.21.15.93:0/3423510425 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f29380058a0 con 0x7f29401049a0 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.055+0000 7f29367fc700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f292c0a1260 0x7f292c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.056+0000 7f2945638700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f292c0a1260 0x7f292c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.056+0000 7f29367fc700 1 -- 172.21.15.93:0/3423510425 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f29380cc670 con 0x7f29401049a0 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.056+0000 7f29367fc700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] conn(0x7f292c0abdd0 0x7f292c0ae200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.056+0000 7f29367fc700 1 -- 172.21.15.93:0/3423510425 --> [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f292c0ae8d0 con 0x7f292c0abdd0 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.056+0000 7f29367fc700 1 -- 172.21.15.93:0/3423510425 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29380cc9e0 con 0x7f29401049a0 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.056+0000 7f2945638700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f292c0a1260 0x7f292c0a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f293c005fd0 tx=0x7f293c005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.056+0000 7f2944e37700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] conn(0x7f292c0abdd0 0x7f292c0ae200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.056+0000 7f2944e37700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] conn(0x7f292c0abdd0 0x7f292c0ae200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:20.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.057+0000 7f29367fc700 1 -- 172.21.15.93:0/3423510425 <== osd.6 v2:172.21.15.93:6818/848954134 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f292c0ae8d0 con 0x7f292c0abdd0 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.073+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 --> [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f294010f3b0 con 0x7f292c0abdd0 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.073+0000 7f29367fc700 1 -- 172.21.15.93:0/3423510425 <== osd.6 v2:172.21.15.93:6818/848954134 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f294010f3b0 con 0x7f292c0abdd0 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] conn(0x7f292c0abdd0 msgr2=0x7f292c0ae200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] conn(0x7f292c0abdd0 0x7f292c0ae200 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f292c0a1260 msgr2=0x7f292c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f292c0a1260 0x7f292c0a3710 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f293c005fd0 tx=0x7f293c005e50 comp rx=0 tx=0).stop 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29401049a0 msgr2=0x7f294019ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 --2- 172.21.15.93:0/3423510425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f29401049a0 0x7f294019ce90 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f2938007ee0 tx=0x7f293800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f2945e39700 1 -- 172.21.15.93:0/3423510425 reap_dead start 2023-12-14T18:25:20.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 shutdown_connections 2023-12-14T18:25:20.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 >> 172.21.15.93:0/3423510425 conn(0x7f29400758a0 msgr2=0x7f29400fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:20.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 shutdown_connections 2023-12-14T18:25:20.225 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:20.074+0000 7f294789c700 1 -- 172.21.15.93:0/3423510425 wait complete. 2023-12-14T18:25:21.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.000+0000 7f70aa6ed700 1 -- 172.21.15.93:0/19292807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a40ff470 msgr2=0x7f70a40ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.000+0000 7f70aa6ed700 1 --2- 172.21.15.93:0/19292807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a40ff470 0x7f70a40ff840 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f70940099c0 tx=0x7f7094009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.000+0000 7f70aa6ed700 1 -- 172.21.15.93:0/19292807 shutdown_connections 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.000+0000 7f70aa6ed700 1 --2- 172.21.15.93:0/19292807 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a40ff470 0x7f70a40ff840 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.000+0000 7f70aa6ed700 1 --2- 172.21.15.93:0/19292807 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70a4104510 0x7f70a410c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.000+0000 7f70aa6ed700 1 --2- 172.21.15.93:0/19292807 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ffe10 0x7f70a4103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.000+0000 7f70aa6ed700 1 -- 172.21.15.93:0/19292807 >> 172.21.15.93:0/19292807 conn(0x7f70a40770d0 msgr2=0x7f70a40774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.000+0000 7f70aa6ed700 1 -- 172.21.15.93:0/19292807 shutdown_connections 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.000+0000 7f70aa6ed700 1 -- 172.21.15.93:0/19292807 wait complete. 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.001+0000 7f70aa6ed700 1 Processor -- start 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.001+0000 7f70aa6ed700 1 -- start start 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.001+0000 7f70aa6ed700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70a40ff470 0x7f70a4100e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.001+0000 7f70aa6ed700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ffe10 0x7f70a41013c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.001+0000 7f70aa6ed700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a4104510 0x7f70a4101940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.001+0000 7f70aa6ed700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41a48e0 con 0x7f70a4104510 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.001+0000 7f70aa6ed700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41a4a50 con 0x7f70a40ff470 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.001+0000 7f70aa6ed700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70a41a4c10 con 0x7f70a40ffe10 2023-12-14T18:25:21.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a8c8a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a4104510 0x7f70a4101940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:21.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a8c8a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a4104510 0x7f70a4101940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:59568/0 (socket says 172.21.15.93:59568) 2023-12-14T18:25:21.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a8c8a700 1 -- 172.21.15.93:0/3970340133 learned_addr learned my addr 172.21.15.93:0/3970340133 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:21.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a37fe700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ffe10 0x7f70a41013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:21.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a3fff700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70a40ff470 0x7f70a4100e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:21.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a8c8a700 1 -- 172.21.15.93:0/3970340133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ffe10 msgr2=0x7f70a41013c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a8c8a700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ffe10 0x7f70a41013c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a8c8a700 1 -- 172.21.15.93:0/3970340133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70a40ff470 msgr2=0x7f70a4100e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a8c8a700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70a40ff470 0x7f70a4100e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a8c8a700 1 -- 172.21.15.93:0/3970340133 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70940095f0 con 0x7f70a4104510 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a3fff700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70a40ff470 0x7f70a4100e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a37fe700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70a40ffe10 0x7f70a41013c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a8c8a700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a4104510 0x7f70a4101940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b970 tx=0x7f709c00bd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a17fa700 1 -- 172.21.15.93:0/3970340133 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c00c9b0 con 0x7f70a4104510 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a17fa700 1 -- 172.21.15.93:0/3970340133 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f709c014440 con 0x7f70a4104510 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70a410f340 con 0x7f70a4104510 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70a17fa700 1 -- 172.21.15.93:0/3970340133 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f709c0075d0 con 0x7f70a4104510 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.002+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70a410f1b0 con 0x7f70a4104510 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.003+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f70a4069890 con 0x7f70a4104510 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.004+0000 7f70a17fa700 1 -- 172.21.15.93:0/3970340133 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f709c0145b0 con 0x7f70a4104510 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.005+0000 7f70a17fa700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f708c0a11e0 0x7f708c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.005+0000 7f70a3fff700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f708c0a11e0 0x7f708c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:21.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.005+0000 7f70a17fa700 1 -- 172.21.15.93:0/3970340133 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f709c006e40 con 0x7f70a4104510 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.005+0000 7f70a17fa700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] conn(0x7f708c0abd50 0x7f708c0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.005+0000 7f70a17fa700 1 -- 172.21.15.93:0/3970340133 --> [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f708c0ae850 con 0x7f708c0abd50 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.005+0000 7f70a17fa700 1 -- 172.21.15.93:0/3970340133 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f709c0ccd30 con 0x7f70a4104510 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.005+0000 7f70a37fe700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] conn(0x7f708c0abd50 0x7f708c0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.005+0000 7f70a3fff700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f708c0a11e0 0x7f708c0a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7094000c00 tx=0x7f7094005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.005+0000 7f70a37fe700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] conn(0x7f708c0abd50 0x7f708c0ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.006+0000 7f70a17fa700 1 -- 172.21.15.93:0/3970340133 <== osd.3 v2:172.21.15.93:6810/980379303 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f708c0ae850 con 0x7f708c0abd50 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.023+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 --> [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f70a410f4d0 con 0x7f708c0abd50 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.023+0000 7f70a17fa700 1 -- 172.21.15.93:0/3970340133 <== osd.3 v2:172.21.15.93:6810/980379303 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f70a410f4d0 con 0x7f708c0abd50 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.023+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] conn(0x7f708c0abd50 msgr2=0x7f708c0ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.023+0000 7f70aa6ed700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] conn(0x7f708c0abd50 0x7f708c0ae180 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.023+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f708c0a11e0 msgr2=0x7f708c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.023+0000 7f70aa6ed700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f708c0a11e0 0x7f708c0a3690 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7094000c00 tx=0x7f7094005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.023+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a4104510 msgr2=0x7f70a4101940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.023+0000 7f70aa6ed700 1 --2- 172.21.15.93:0/3970340133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70a4104510 0x7f70a4101940 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f709c00b970 tx=0x7f709c00bd30 comp rx=0 tx=0).stop 2023-12-14T18:25:21.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.023+0000 7f70a8c8a700 1 -- 172.21.15.93:0/3970340133 reap_dead start 2023-12-14T18:25:21.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.024+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 shutdown_connections 2023-12-14T18:25:21.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.024+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 >> 172.21.15.93:0/3970340133 conn(0x7f70a40770d0 msgr2=0x7f70a4068ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:21.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.024+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 shutdown_connections 2023-12-14T18:25:21.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:21.024+0000 7f70aa6ed700 1 -- 172.21.15.93:0/3970340133 wait complete. 2023-12-14T18:25:21.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:21 smithi093 ceph-mon[42070]: pgmap v246: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-12-14T18:25:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:21 smithi097 ceph-mon[43486]: pgmap v246: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-12-14T18:25:21.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:21 smithi160 ceph-mon[43840]: pgmap v246: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-12-14T18:25:21.654 INFO:teuthology.orchestra.run.smithi093.stdout:322122547219 2023-12-14T18:25:21.654 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.11 2023-12-14T18:25:22.033 INFO:teuthology.orchestra.run.smithi093.stdout:103079215150 2023-12-14T18:25:22.033 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.3 2023-12-14T18:25:22.325 INFO:teuthology.orchestra.run.smithi093.stdout:266287972377 2023-12-14T18:25:22.326 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.9 2023-12-14T18:25:22.478 INFO:teuthology.orchestra.run.smithi093.stdout:184683593762 2023-12-14T18:25:22.478 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.6 2023-12-14T18:25:22.731 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:23.056 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 -- 172.21.15.93:0/3423559314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b6c105290 msgr2=0x7f8b6c105660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 --2- 172.21.15.93:0/3423559314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b6c105290 0x7f8b6c105660 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f8b68009990 tx=0x7f8b68009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 -- 172.21.15.93:0/3423559314 shutdown_connections 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 --2- 172.21.15.93:0/3423559314 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b6c105290 0x7f8b6c105660 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 --2- 172.21.15.93:0/3423559314 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b6c0ff1e0 0x7f8b6c10d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 --2- 172.21.15.93:0/3423559314 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6c0fe830 0x7f8b6c0feca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 -- 172.21.15.93:0/3423559314 >> 172.21.15.93:0/3423559314 conn(0x7f8b6c0758a0 msgr2=0x7f8b6c075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 -- 172.21.15.93:0/3423559314 shutdown_connections 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 -- 172.21.15.93:0/3423559314 wait complete. 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 Processor -- start 2023-12-14T18:25:23.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.045+0000 7f8b73959700 1 -- start start 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b73959700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6c0fe830 0x7f8b6c0ffa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b73959700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b6c0ff1e0 0x7f8b6c0fff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b73959700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b6c105290 0x7f8b6c1004d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b73959700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1a4850 con 0x7f8b6c105290 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b73959700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1a49c0 con 0x7f8b6c0fe830 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b73959700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b6c1a4b80 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b70ef4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b6c0ff1e0 0x7f8b6c0fff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b71ef6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b6c105290 0x7f8b6c1004d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b70ef4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b6c0ff1e0 0x7f8b6c0fff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:43378/0 (socket says 172.21.15.93:43378) 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b70ef4700 1 -- 172.21.15.93:0/2667090866 learned_addr learned my addr 172.21.15.93:0/2667090866 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b716f5700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6c0fe830 0x7f8b6c0ffa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b70ef4700 1 -- 172.21.15.93:0/2667090866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6c0fe830 msgr2=0x7f8b6c0ffa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b70ef4700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8b6c0fe830 0x7f8b6c0ffa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b70ef4700 1 -- 172.21.15.93:0/2667090866 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b6c105290 msgr2=0x7f8b6c1004d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b70ef4700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8b6c105290 0x7f8b6c1004d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.211 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.046+0000 7f8b70ef4700 1 -- 172.21.15.93:0/2667090866 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b680095f0 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.047+0000 7f8b70ef4700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b6c0ff1e0 0x7f8b6c0fff50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00d950 tx=0x7f8b5c00dc60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.047+0000 7f8b627fc700 1 -- 172.21.15.93:0/2667090866 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c009940 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.047+0000 7f8b627fc700 1 -- 172.21.15.93:0/2667090866 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8b5c010460 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.047+0000 7f8b627fc700 1 -- 172.21.15.93:0/2667090866 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c00f620 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.047+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b6c110410 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.047+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6c110280 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.048+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8b6c0fccd0 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.051+0000 7f8b627fc700 1 -- 172.21.15.93:0/2667090866 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8b5c0105d0 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.051+0000 7f8b627fc700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b580a12e0 0x7f8b580a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.052+0000 7f8b627fc700 1 -- 172.21.15.93:0/2667090866 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f8b5c0ccf10 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.052+0000 7f8b716f5700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b580a12e0 0x7f8b580a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.052+0000 7f8b627fc700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] conn(0x7f8b580abe50 0x7f8b580ae280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.052+0000 7f8b627fc700 1 -- 172.21.15.93:0/2667090866 --> [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8b580ae950 con 0x7f8b580abe50 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.052+0000 7f8b627fc700 1 -- 172.21.15.93:0/2667090866 <== mon.1 v2:172.21.15.160:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8b5c0cd2c0 con 0x7f8b6c0ff1e0 2023-12-14T18:25:23.212 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.052+0000 7f8b71ef6700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] conn(0x7f8b580abe50 0x7f8b580ae280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.052+0000 7f8b716f5700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b580a12e0 0x7f8b580a3790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800b5c0 tx=0x7f8b68005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.053+0000 7f8b71ef6700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] conn(0x7f8b580abe50 0x7f8b580ae280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.053+0000 7f8b627fc700 1 -- 172.21.15.93:0/2667090866 <== osd.5 v2:172.21.15.160:6808/1570120079 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f8b580ae950 con 0x7f8b580abe50 2023-12-14T18:25:23.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.070+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 --> [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8b6c1105a0 con 0x7f8b580abe50 2023-12-14T18:25:23.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b627fc700 1 -- 172.21.15.93:0/2667090866 <== osd.5 v2:172.21.15.160:6808/1570120079 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8b6c1105a0 con 0x7f8b580abe50 2023-12-14T18:25:23.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 >> [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] conn(0x7f8b580abe50 msgr2=0x7f8b580ae280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b73959700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] conn(0x7f8b580abe50 0x7f8b580ae280 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b580a12e0 msgr2=0x7f8b580a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.213 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b73959700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8b580a12e0 0x7f8b580a3790 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8b6800b5c0 tx=0x7f8b68005fd0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b6c0ff1e0 msgr2=0x7f8b6c0fff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b73959700 1 --2- 172.21.15.93:0/2667090866 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8b6c0ff1e0 0x7f8b6c0fff50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c00d950 tx=0x7f8b5c00dc60 comp rx=0 tx=0).stop 2023-12-14T18:25:23.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b71ef6700 1 -- 172.21.15.93:0/2667090866 reap_dead start 2023-12-14T18:25:23.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 shutdown_connections 2023-12-14T18:25:23.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.071+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 >> 172.21.15.93:0/2667090866 conn(0x7f8b6c0758a0 msgr2=0x7f8b6c0fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:23.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.072+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 shutdown_connections 2023-12-14T18:25:23.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.072+0000 7f8b73959700 1 -- 172.21.15.93:0/2667090866 wait complete. 2023-12-14T18:25:23.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:23 smithi093 ceph-mon[42070]: pgmap v247: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-12-14T18:25:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:23 smithi097 ceph-mon[43486]: pgmap v247: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-12-14T18:25:23.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.292+0000 7f15a9943700 1 -- 172.21.15.93:0/3982036078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15a4100540 msgr2=0x7f15a4103e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.292+0000 7f15a9943700 1 --2- 172.21.15.93:0/3982036078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15a4100540 0x7f15a4103e40 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f15980099c0 tx=0x7f1598009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.292+0000 7f15a9943700 1 -- 172.21.15.93:0/3982036078 shutdown_connections 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.292+0000 7f15a9943700 1 --2- 172.21.15.93:0/3982036078 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15a4104570 0x7f15a410c130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.292+0000 7f15a9943700 1 --2- 172.21.15.93:0/3982036078 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15a40ffba0 0x7f15a40fff70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.292+0000 7f15a9943700 1 --2- 172.21.15.93:0/3982036078 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15a4100540 0x7f15a4103e40 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.292+0000 7f15a9943700 1 -- 172.21.15.93:0/3982036078 >> 172.21.15.93:0/3982036078 conn(0x7f15a40758a0 msgr2=0x7f15a4075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.292+0000 7f15a9943700 1 -- 172.21.15.93:0/3982036078 shutdown_connections 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.292+0000 7f15a9943700 1 -- 172.21.15.93:0/3982036078 wait complete. 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a9943700 1 Processor -- start 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a9943700 1 -- start start 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a9943700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15a40ffba0 0x7f15a4197210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a9943700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15a4100540 0x7f15a4197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a9943700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15a4104570 0x7f15a41a2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a9943700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a4198090 con 0x7f15a4104570 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a9943700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41a27d0 con 0x7f15a40ffba0 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a9943700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15a41a2990 con 0x7f15a4100540 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15a4104570 0x7f15a41a2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15a4104570 0x7f15a41a2290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48668/0 (socket says 172.21.15.93:48668) 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a37fe700 1 -- 172.21.15.93:0/3526538808 learned_addr learned my addr 172.21.15.93:0/3526538808 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a27fc700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15a4100540 0x7f15a4197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a2ffd700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15a40ffba0 0x7f15a4197210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a37fe700 1 -- 172.21.15.93:0/3526538808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15a4100540 msgr2=0x7f15a4197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a37fe700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f15a4100540 0x7f15a4197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a37fe700 1 -- 172.21.15.93:0/3526538808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15a40ffba0 msgr2=0x7f15a4197210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a37fe700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f15a40ffba0 0x7f15a4197210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.293+0000 7f15a37fe700 1 -- 172.21.15.93:0/3526538808 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15980095f0 con 0x7f15a4104570 2023-12-14T18:25:23.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.294+0000 7f15a37fe700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15a4104570 0x7f15a41a2290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f159c007ee0 tx=0x7f159c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.294+0000 7f15a8941700 1 -- 172.21.15.93:0/3526538808 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c017070 con 0x7f15a4104570 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.294+0000 7f15a8941700 1 -- 172.21.15.93:0/3526538808 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f159c00f040 con 0x7f15a4104570 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.294+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15a410f2e0 con 0x7f15a4104570 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.294+0000 7f15a8941700 1 -- 172.21.15.93:0/3526538808 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f159c005580 con 0x7f15a4104570 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.294+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15a410f150 con 0x7f15a4104570 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.295+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f15a40fce30 con 0x7f15a4104570 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.296+0000 7f15a8941700 1 -- 172.21.15.93:0/3526538808 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f159c00db20 con 0x7f15a4104570 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.297+0000 7f15a8941700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f158c0a11e0 0x7f158c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.297+0000 7f15a2ffd700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f158c0a11e0 0x7f158c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.297+0000 7f15a8941700 1 -- 172.21.15.93:0/3526538808 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f159c0cca50 con 0x7f15a4104570 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.297+0000 7f15a8941700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] conn(0x7f158c0abd50 0x7f158c0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.297+0000 7f15a8941700 1 -- 172.21.15.93:0/3526538808 --> [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f158c0ae850 con 0x7f158c0abd50 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.297+0000 7f15a8941700 1 -- 172.21.15.93:0/3526538808 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f159c0cce60 con 0x7f15a4104570 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.297+0000 7f15a27fc700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] conn(0x7f158c0abd50 0x7f158c0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.297+0000 7f15a2ffd700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f158c0a11e0 0x7f158c0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1594009c60 tx=0x7f1594009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.297+0000 7f15a27fc700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] conn(0x7f158c0abd50 0x7f158c0ae180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.298+0000 7f15a8941700 1 -- 172.21.15.93:0/3526538808 <== osd.0 v2:172.21.15.93:6802/822854131 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f158c0ae850 con 0x7f158c0abd50 2023-12-14T18:25:23.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.314+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 --> [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f15a410f470 con 0x7f158c0abd50 2023-12-14T18:25:23.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a8941700 1 -- 172.21.15.93:0/3526538808 <== osd.0 v2:172.21.15.93:6802/822854131 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f15a410f470 con 0x7f158c0abd50 2023-12-14T18:25:23.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] conn(0x7f158c0abd50 msgr2=0x7f158c0ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] conn(0x7f158c0abd50 0x7f158c0ae180 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f158c0a11e0 msgr2=0x7f158c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f158c0a11e0 0x7f158c0a3690 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f1594009c60 tx=0x7f1594009260 comp rx=0 tx=0).stop 2023-12-14T18:25:23.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15a4104570 msgr2=0x7f15a41a2290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 --2- 172.21.15.93:0/3526538808 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f15a4104570 0x7f15a41a2290 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f159c007ee0 tx=0x7f159c00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a37fe700 1 -- 172.21.15.93:0/3526538808 reap_dead start 2023-12-14T18:25:23.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 shutdown_connections 2023-12-14T18:25:23.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 >> 172.21.15.93:0/3526538808 conn(0x7f15a40758a0 msgr2=0x7f15a40fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:23.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 shutdown_connections 2023-12-14T18:25:23.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.315+0000 7f15a9943700 1 -- 172.21.15.93:0/3526538808 wait complete. 2023-12-14T18:25:23.366 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.395+0000 7fda3a41f700 1 -- 172.21.15.93:0/2481133233 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda34104500 msgr2=0x7fda3410c080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.395+0000 7fda3a41f700 1 --2- 172.21.15.93:0/2481133233 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda34104500 0x7fda3410c080 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fda2c00b0d0 tx=0x7fda2c00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.395+0000 7fda3a41f700 1 -- 172.21.15.93:0/2481133233 shutdown_connections 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.395+0000 7fda3a41f700 1 --2- 172.21.15.93:0/2481133233 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda34104500 0x7fda3410c080 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.395+0000 7fda3a41f700 1 --2- 172.21.15.93:0/2481133233 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda340ff460 0x7fda340ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.395+0000 7fda3a41f700 1 --2- 172.21.15.93:0/2481133233 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda340ffe00 0x7fda34103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.395+0000 7fda3a41f700 1 -- 172.21.15.93:0/2481133233 >> 172.21.15.93:0/2481133233 conn(0x7fda340770a0 msgr2=0x7fda340774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.395+0000 7fda3a41f700 1 -- 172.21.15.93:0/2481133233 shutdown_connections 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.395+0000 7fda3a41f700 1 -- 172.21.15.93:0/2481133233 wait complete. 2023-12-14T18:25:23.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.396+0000 7fda3a41f700 1 Processor -- start 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.396+0000 7fda3a41f700 1 -- start start 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.396+0000 7fda3a41f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda340ff460 0x7fda3406fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.396+0000 7fda3a41f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda340ffe00 0x7fda34070330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda3a41f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda34104500 0x7fda3406af60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda3a41f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3406b670 con 0x7fda340ffe00 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda3a41f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3406b7e0 con 0x7fda340ff460 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda3a41f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda3406b9a0 con 0x7fda34104500 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda337fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda340ffe00 0x7fda34070330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda337fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda340ffe00 0x7fda34070330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48694/0 (socket says 172.21.15.93:48694) 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda337fe700 1 -- 172.21.15.93:0/754400134 learned_addr learned my addr 172.21.15.93:0/754400134 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda33fff700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda340ff460 0x7fda3406fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda389bc700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda34104500 0x7fda3406af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda337fe700 1 -- 172.21.15.93:0/754400134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda34104500 msgr2=0x7fda3406af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda337fe700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fda34104500 0x7fda3406af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda337fe700 1 -- 172.21.15.93:0/754400134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda340ff460 msgr2=0x7fda3406fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda337fe700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fda340ff460 0x7fda3406fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda337fe700 1 -- 172.21.15.93:0/754400134 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda2c009d00 con 0x7fda340ffe00 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda337fe700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda340ffe00 0x7fda34070330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fda2800d950 tx=0x7fda2800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda317fa700 1 -- 172.21.15.93:0/754400134 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28004220 con 0x7fda340ffe00 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.397+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda341103f0 con 0x7fda340ffe00 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.398+0000 7fda317fa700 1 -- 172.21.15.93:0/754400134 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fda28007620 con 0x7fda340ffe00 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.398+0000 7fda317fa700 1 -- 172.21.15.93:0/754400134 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda28005690 con 0x7fda340ffe00 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.398+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda34110260 con 0x7fda340ffe00 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.398+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fda340fd5d0 con 0x7fda340ffe00 2023-12-14T18:25:23.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.400+0000 7fda317fa700 1 -- 172.21.15.93:0/754400134 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fda28004750 con 0x7fda340ffe00 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.400+0000 7fda317fa700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fda1c0a11e0 0x7fda1c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.400+0000 7fda317fa700 1 -- 172.21.15.93:0/754400134 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fda280cd070 con 0x7fda340ffe00 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.400+0000 7fda33fff700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fda1c0a11e0 0x7fda1c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.401+0000 7fda317fa700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] conn(0x7fda1c0abd50 0x7fda1c0ae180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.401+0000 7fda317fa700 1 -- 172.21.15.93:0/754400134 --> [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fda1c0ae850 con 0x7fda1c0abd50 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.401+0000 7fda317fa700 1 -- 172.21.15.93:0/754400134 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7fda280cd470 con 0x7fda340ffe00 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.401+0000 7fda33fff700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fda1c0a11e0 0x7fda1c0a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda24009c60 tx=0x7fda24009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.401+0000 7fda389bc700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] conn(0x7fda1c0abd50 0x7fda1c0ae180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.401+0000 7fda389bc700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] conn(0x7fda1c0abd50 0x7fda1c0ae180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.402+0000 7fda317fa700 1 -- 172.21.15.93:0/754400134 <== osd.2 v2:172.21.15.160:6800/3346159121 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fda1c0ae850 con 0x7fda1c0abd50 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.418+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 --> [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fda34110580 con 0x7fda1c0abd50 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.418+0000 7fda317fa700 1 -- 172.21.15.93:0/754400134 <== osd.2 v2:172.21.15.160:6800/3346159121 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fda34110580 con 0x7fda1c0abd50 2023-12-14T18:25:23.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 >> [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] conn(0x7fda1c0abd50 msgr2=0x7fda1c0ae180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] conn(0x7fda1c0abd50 0x7fda1c0ae180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fda1c0a11e0 msgr2=0x7fda1c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fda1c0a11e0 0x7fda1c0a3690 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fda24009c60 tx=0x7fda24009260 comp rx=0 tx=0).stop 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda340ffe00 msgr2=0x7fda34070330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 --2- 172.21.15.93:0/754400134 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fda340ffe00 0x7fda34070330 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fda2800d950 tx=0x7fda2800dc60 comp rx=0 tx=0).stop 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda389bc700 1 -- 172.21.15.93:0/754400134 reap_dead start 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 shutdown_connections 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 >> 172.21.15.93:0/754400134 conn(0x7fda340770a0 msgr2=0x7fda340fcc00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 shutdown_connections 2023-12-14T18:25:23.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:23.419+0000 7fda3a41f700 1 -- 172.21.15.93:0/754400134 wait complete. 2023-12-14T18:25:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:23 smithi160 ceph-mon[43840]: pgmap v247: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-12-14T18:25:23.740 INFO:teuthology.orchestra.run.smithi093.stdout:154618822696 2023-12-14T18:25:23.740 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.5 2023-12-14T18:25:24.019 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.017+0000 7f8128a26700 1 -- 172.21.15.93:0/2296115916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81241041c0 msgr2=0x7f8124104590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:24.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.017+0000 7f8128a26700 1 --2- 172.21.15.93:0/2296115916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81241041c0 0x7f8124104590 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f81140099c0 tx=0x7f8114009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:24.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.018+0000 7f8128a26700 1 -- 172.21.15.93:0/2296115916 shutdown_connections 2023-12-14T18:25:24.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.018+0000 7f8128a26700 1 --2- 172.21.15.93:0/2296115916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81241041c0 0x7f8124104590 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:24.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.018+0000 7f8128a26700 1 --2- 172.21.15.93:0/2296115916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8124100330 0x7f812410c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:24.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.018+0000 7f8128a26700 1 --2- 172.21.15.93:0/2296115916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f81240ff980 0x7f81240ffdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:24.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.018+0000 7f8128a26700 1 -- 172.21.15.93:0/2296115916 >> 172.21.15.93:0/2296115916 conn(0x7f81240758a0 msgr2=0x7f8124075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:24.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.018+0000 7f8128a26700 1 -- 172.21.15.93:0/2296115916 shutdown_connections 2023-12-14T18:25:24.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.018+0000 7f8128a26700 1 -- 172.21.15.93:0/2296115916 wait complete. 2023-12-14T18:25:24.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.018+0000 7f8128a26700 1 Processor -- start 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8128a26700 1 -- start start 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8128a26700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81240ff980 0x7f812406af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8128a26700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8124100330 0x7f812406b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8128a26700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81241041c0 0x7f81241a8760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8128a26700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81241a8ca0 con 0x7f81241041c0 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8128a26700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81241a8e10 con 0x7f8124100330 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8128a26700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81241ad3e0 con 0x7f81240ff980 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8122d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81241041c0 0x7f81241a8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8122d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81241041c0 0x7f81241a8760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48708/0 (socket says 172.21.15.93:48708) 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f812259c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81240ff980 0x7f812406af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8122d9d700 1 -- 172.21.15.93:0/1904154595 learned_addr learned my addr 172.21.15.93:0/1904154595 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8121d9b700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8124100330 0x7f812406b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8122d9d700 1 -- 172.21.15.93:0/1904154595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81240ff980 msgr2=0x7f812406af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8122d9d700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81240ff980 0x7f812406af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:24.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8122d9d700 1 -- 172.21.15.93:0/1904154595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8124100330 msgr2=0x7f812406b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:24.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8122d9d700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8124100330 0x7f812406b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:24.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8122d9d700 1 -- 172.21.15.93:0/1904154595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81140095f0 con 0x7f81241041c0 2023-12-14T18:25:24.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.019+0000 7f8122d9d700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81241041c0 0x7f81241a8760 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f811c00ef30 tx=0x7f811c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:24.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.020+0000 7f81137fe700 1 -- 172.21.15.93:0/1904154595 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c018070 con 0x7f81241041c0 2023-12-14T18:25:24.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.020+0000 7f81137fe700 1 -- 172.21.15.93:0/1904154595 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f811c00f040 con 0x7f81241041c0 2023-12-14T18:25:24.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.020+0000 7f81137fe700 1 -- 172.21.15.93:0/1904154595 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c00cd00 con 0x7f81241041c0 2023-12-14T18:25:24.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.020+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f812410e1d0 con 0x7f81241041c0 2023-12-14T18:25:24.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.020+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f812410e040 con 0x7f81241041c0 2023-12-14T18:25:24.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.020+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f81240fccd0 con 0x7f81241041c0 2023-12-14T18:25:24.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.022+0000 7f81137fe700 1 -- 172.21.15.93:0/1904154595 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f811c007500 con 0x7f81241041c0 2023-12-14T18:25:24.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.023+0000 7f81137fe700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f810c0a1340 0x7f810c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:24.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.023+0000 7f812259c700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f810c0a1340 0x7f810c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:24.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.023+0000 7f81137fe700 1 -- 172.21.15.93:0/1904154595 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f811c0ccfd0 con 0x7f81241041c0 2023-12-14T18:25:24.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.023+0000 7f81137fe700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] conn(0x7f810c0abeb0 0x7f810c0ae2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:24.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.023+0000 7f81137fe700 1 -- 172.21.15.93:0/1904154595 --> [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f810c0ae9b0 con 0x7f810c0abeb0 2023-12-14T18:25:24.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.023+0000 7f81137fe700 1 -- 172.21.15.93:0/1904154595 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_get_version_reply(handle=1 version=84) v2 ==== 24+0+0 (secure 0 0 0) 0x7f811c0cd340 con 0x7f81241041c0 2023-12-14T18:25:24.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.023+0000 7f812259c700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f810c0a1340 0x7f810c0a37f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8114009fd0 tx=0x7f8114005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:24.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.023+0000 7f8121d9b700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] conn(0x7f810c0abeb0 0x7f810c0ae2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:24.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.024+0000 7f8121d9b700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] conn(0x7f810c0abeb0 0x7f810c0ae2e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:24.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.024+0000 7f81137fe700 1 -- 172.21.15.93:0/1904154595 <== osd.10 v2:172.21.15.97:6824/240921879 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f810c0ae9b0 con 0x7f810c0abeb0 2023-12-14T18:25:24.042 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.040+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 --> [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f812410e360 con 0x7f810c0abeb0 2023-12-14T18:25:24.043 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.041+0000 7f81137fe700 1 -- 172.21.15.93:0/1904154595 <== osd.10 v2:172.21.15.97:6824/240921879 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f812410e360 con 0x7f810c0abeb0 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 >> [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] conn(0x7f810c0abeb0 msgr2=0x7f810c0ae2e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] conn(0x7f810c0abeb0 0x7f810c0ae2e0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f810c0a1340 msgr2=0x7f810c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f810c0a1340 0x7f810c0a37f0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f8114009fd0 tx=0x7f8114005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81241041c0 msgr2=0x7f81241a8760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 --2- 172.21.15.93:0/1904154595 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81241041c0 0x7f81241a8760 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f811c00ef30 tx=0x7f811c00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8122d9d700 1 -- 172.21.15.93:0/1904154595 reap_dead start 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 shutdown_connections 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 >> 172.21.15.93:0/1904154595 conn(0x7f81240758a0 msgr2=0x7f81240fc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 shutdown_connections 2023-12-14T18:25:24.044 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:24.042+0000 7f8128a26700 1 -- 172.21.15.93:0/1904154595 wait complete. 2023-12-14T18:25:25.224 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:25 smithi093 ceph-mon[42070]: pgmap v248: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-12-14T18:25:25.242 INFO:teuthology.orchestra.run.smithi093.stdout:77309411378 2023-12-14T18:25:25.242 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.2 2023-12-14T18:25:25.270 INFO:teuthology.orchestra.run.smithi093.stdout:287762808855 2023-12-14T18:25:25.271 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.10 2023-12-14T18:25:25.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:25 smithi097 ceph-mon[43486]: pgmap v248: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-12-14T18:25:25.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:25 smithi160 ceph-mon[43840]: pgmap v248: 129 pgs: 129 active+clean; 635 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2023-12-14T18:25:25.578 INFO:teuthology.orchestra.run.smithi093.stdout:34359738426 2023-12-14T18:25:25.578 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph osd last-stat-seq osd.0 2023-12-14T18:25:25.889 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:26.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.467+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2772393288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6cc100540 msgr2=0x7fb6cc100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.467+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2772393288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6cc100540 0x7fb6cc100910 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc0099c0 tx=0x7fb6bc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.467+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2772393288 shutdown_connections 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.467+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2772393288 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6cc100540 0x7fb6cc100910 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.467+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2772393288 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6cc1055e0 0x7fb6cc10d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.467+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2772393288 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc100ee0 0x7fb6cc104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.467+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2772393288 >> 172.21.15.93:0/2772393288 conn(0x7fb6cc0fbdc0 msgr2=0x7fb6cc0fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.467+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2772393288 shutdown_connections 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.467+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2772393288 wait complete. 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.468+0000 7fb6d14c7700 1 Processor -- start 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.468+0000 7fb6d14c7700 1 -- start start 2023-12-14T18:25:26.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.468+0000 7fb6d14c7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6cc100540 0x7fb6cc196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.468+0000 7fb6d14c7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6cc100ee0 0x7fb6cc196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.468+0000 7fb6d14c7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc1055e0 0x7fb6cc19add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.468+0000 7fb6d14c7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc19b3e0 con 0x7fb6cc100ee0 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.468+0000 7fb6d14c7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc19b550 con 0x7fb6cc100540 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.468+0000 7fb6d14c7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc19b710 con 0x7fb6cc1055e0 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6caffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6cc100540 0x7fb6cc196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6ca7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6cc100ee0 0x7fb6cc196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6caffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6cc100540 0x7fb6cc196420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:60162/0 (socket says 172.21.15.93:60162) 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6caffd700 1 -- 172.21.15.93:0/2928677168 learned_addr learned my addr 172.21.15.93:0/2928677168 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6cb7fe700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc1055e0 0x7fb6cc19add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6caffd700 1 -- 172.21.15.93:0/2928677168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc1055e0 msgr2=0x7fb6cc19add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6caffd700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc1055e0 0x7fb6cc19add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6caffd700 1 -- 172.21.15.93:0/2928677168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6cc100ee0 msgr2=0x7fb6cc196960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6caffd700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6cc100ee0 0x7fb6cc196960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6caffd700 1 -- 172.21.15.93:0/2928677168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6bc0095f0 con 0x7fb6cc100540 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6cb7fe700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc1055e0 0x7fb6cc19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6caffd700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6cc100540 0x7fb6cc196420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00b580 tx=0x7fb6bc00f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6b3fff700 1 -- 172.21.15.93:0/2928677168 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc01d070 con 0x7fb6cc100540 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc110320 con 0x7fb6cc100540 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.469+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc110190 con 0x7fb6cc100540 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.470+0000 7fb6b3fff700 1 -- 172.21.15.93:0/2928677168 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb6bc00fb80 con 0x7fb6cc100540 2023-12-14T18:25:26.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.470+0000 7fb6b3fff700 1 -- 172.21.15.93:0/2928677168 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6bc0173f0 con 0x7fb6cc100540 2023-12-14T18:25:26.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.471+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6b8005320 con 0x7fb6cc100540 2023-12-14T18:25:26.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.473+0000 7fb6b3fff700 1 -- 172.21.15.93:0/2928677168 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb6bc022470 con 0x7fb6cc100540 2023-12-14T18:25:26.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.473+0000 7fb6b3fff700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb6b40a12e0 0x7fb6b40a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.473+0000 7fb6b3fff700 1 -- 172.21.15.93:0/2928677168 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fb6bc0cdc70 con 0x7fb6cc100540 2023-12-14T18:25:26.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.473+0000 7fb6ca7fc700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb6b40a12e0 0x7fb6b40a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.474+0000 7fb6ca7fc700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb6b40a12e0 0x7fb6b40a3790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc197a40 tx=0x7fb6c000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:26.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.478+0000 7fb6b3fff700 1 -- 172.21.15.93:0/2928677168 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb6bc05c730 con 0x7fb6cc100540 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 -- 172.21.15.93:0/1162540269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f8072e10 msgr2=0x7f58f80731e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 --2- 172.21.15.93:0/1162540269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f8072e10 0x7f58f80731e0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f58ec009990 tx=0x7f58ec009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 -- 172.21.15.93:0/1162540269 shutdown_connections 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 --2- 172.21.15.93:0/1162540269 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f8072e10 0x7f58f80731e0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 --2- 172.21.15.93:0/1162540269 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f810cbc0 0x7f58f810efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 --2- 172.21.15.93:0/1162540269 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8073720 0x7f58f810c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 -- 172.21.15.93:0/1162540269 >> 172.21.15.93:0/1162540269 conn(0x7f58f80fbde0 msgr2=0x7f58f80fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 -- 172.21.15.93:0/1162540269 shutdown_connections 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 -- 172.21.15.93:0/1162540269 wait complete. 2023-12-14T18:25:26.568 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 Processor -- start 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 -- start start 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f8072e10 0x7f58f8198480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8073720 0x7f58f81989c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f810cbc0 0x7f58f819ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f819d440 con 0x7f58f810cbc0 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f819d5b0 con 0x7f58f8072e10 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.446+0000 7f58fee89700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f819d770 con 0x7f58f8073720 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fcc25700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f8072e10 0x7f58f8198480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fd426700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f810cbc0 0x7f58f819ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fd426700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f810cbc0 0x7f58f819ce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48722/0 (socket says 172.21.15.93:48722) 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fcc25700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f8072e10 0x7f58f8198480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:60144/0 (socket says 172.21.15.93:60144) 2023-12-14T18:25:26.569 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fd426700 1 -- 172.21.15.93:0/1550478266 learned_addr learned my addr 172.21.15.93:0/1550478266 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58f7fff700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8073720 0x7f58f81989c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fd426700 1 -- 172.21.15.93:0/1550478266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8073720 msgr2=0x7f58f81989c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fd426700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8073720 0x7f58f81989c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fd426700 1 -- 172.21.15.93:0/1550478266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f8072e10 msgr2=0x7f58f8198480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fd426700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f8072e10 0x7f58f8198480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fd426700 1 -- 172.21.15.93:0/1550478266 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0095f0 con 0x7f58f810cbc0 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fd426700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f810cbc0 0x7f58f819ce30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007d70 tx=0x7f58f000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58f5ffb700 1 -- 172.21.15.93:0/1550478266 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f000ad40 con 0x7f58f810cbc0 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58f5ffb700 1 -- 172.21.15.93:0/1550478266 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f58f00128e0 con 0x7f58f810cbc0 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58f5ffb700 1 -- 172.21.15.93:0/1550478266 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f00116b0 con 0x7f58f810cbc0 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f81124f0 con 0x7f58f810cbc0 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.447+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f8112360 con 0x7f58f810cbc0 2023-12-14T18:25:26.570 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.448+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f8112680 con 0x7f58f810cbc0 2023-12-14T18:25:26.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.449+0000 7f58f5ffb700 1 -- 172.21.15.93:0/1550478266 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f58f0012420 con 0x7f58f810cbc0 2023-12-14T18:25:26.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.450+0000 7f58f5ffb700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58e80a11e0 0x7f58e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.450+0000 7f58fcc25700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58e80a11e0 0x7f58e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.451+0000 7f58f5ffb700 1 -- 172.21.15.93:0/1550478266 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f58f00ccda0 con 0x7f58f810cbc0 2023-12-14T18:25:26.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.451+0000 7f58fcc25700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58e80a11e0 0x7f58e80a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b5c0 tx=0x7f58ec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:26.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.455+0000 7f58f5ffb700 1 -- 172.21.15.93:0/1550478266 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f58f00962b0 con 0x7f58f810cbc0 2023-12-14T18:25:26.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.635+0000 7fd68db2e700 1 -- 172.21.15.93:0/2721938141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ff460 msgr2=0x7fd6880ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.635+0000 7fd68db2e700 1 --2- 172.21.15.93:0/2721938141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ff460 0x7fd6880ff830 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fd6780099c0 tx=0x7fd678009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.635+0000 7fd68db2e700 1 -- 172.21.15.93:0/2721938141 shutdown_connections 2023-12-14T18:25:26.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.635+0000 7fd68db2e700 1 --2- 172.21.15.93:0/2721938141 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ff460 0x7fd6880ff830 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.635+0000 7fd68db2e700 1 --2- 172.21.15.93:0/2721938141 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd688104500 0x7fd68810c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.635+0000 7fd68db2e700 1 --2- 172.21.15.93:0/2721938141 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd6880ffe00 0x7fd688103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.637 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.635+0000 7fd68db2e700 1 -- 172.21.15.93:0/2721938141 >> 172.21.15.93:0/2721938141 conn(0x7fd6880770a0 msgr2=0x7fd6880774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:26.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.635+0000 7fd68db2e700 1 -- 172.21.15.93:0/2721938141 shutdown_connections 2023-12-14T18:25:26.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.635+0000 7fd68db2e700 1 -- 172.21.15.93:0/2721938141 wait complete. 2023-12-14T18:25:26.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd68db2e700 1 Processor -- start 2023-12-14T18:25:26.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd68db2e700 1 -- start start 2023-12-14T18:25:26.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd68db2e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6880ff460 0x7fd6881964e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd68db2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ffe00 0x7fd688196a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd68db2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd688104500 0x7fd68819ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.639 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd68db2e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68819b430 con 0x7fd6880ffe00 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd68db2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68819b5a0 con 0x7fd688104500 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd68db2e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd68819b760 con 0x7fd6880ff460 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd686ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ffe00 0x7fd688196a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd686ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ffe00 0x7fd688196a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48748/0 (socket says 172.21.15.93:48748) 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd686ffd700 1 -- 172.21.15.93:0/959453332 learned_addr learned my addr 172.21.15.93:0/959453332 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd687fff700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd688104500 0x7fd68819ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.636+0000 7fd6877fe700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6880ff460 0x7fd6881964e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd686ffd700 1 -- 172.21.15.93:0/959453332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6880ff460 msgr2=0x7fd6881964e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd686ffd700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6880ff460 0x7fd6881964e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd686ffd700 1 -- 172.21.15.93:0/959453332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd688104500 msgr2=0x7fd68819ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd686ffd700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd688104500 0x7fd68819ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd686ffd700 1 -- 172.21.15.93:0/959453332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6780095f0 con 0x7fd6880ffe00 2023-12-14T18:25:26.640 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd687fff700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd688104500 0x7fd68819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:26.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd686ffd700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ffe00 0x7fd688196a20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00c8f0 tx=0x7fd67c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:26.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd684ff9700 1 -- 172.21.15.93:0/959453332 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c007b00 con 0x7fd6880ffe00 2023-12-14T18:25:26.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd684ff9700 1 -- 172.21.15.93:0/959453332 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd67c007c60 con 0x7fd6880ffe00 2023-12-14T18:25:26.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd68810f290 con 0x7fd6880ffe00 2023-12-14T18:25:26.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd684ff9700 1 -- 172.21.15.93:0/959453332 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd67c018660 con 0x7fd6880ffe00 2023-12-14T18:25:26.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.637+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd68810f100 con 0x7fd6880ffe00 2023-12-14T18:25:26.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.638+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd68810f420 con 0x7fd6880ffe00 2023-12-14T18:25:26.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.640+0000 7fd684ff9700 1 -- 172.21.15.93:0/959453332 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd67c00f450 con 0x7fd6880ffe00 2023-12-14T18:25:26.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.641+0000 7fd684ff9700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd6700a1260 0x7fd6700a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:26.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.641+0000 7fd6877fe700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd6700a1260 0x7fd6700a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:26.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.641+0000 7fd684ff9700 1 -- 172.21.15.93:0/959453332 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd67c0d33b0 con 0x7fd6880ffe00 2023-12-14T18:25:26.643 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.641+0000 7fd6877fe700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd6700a1260 0x7fd6700a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd678000c00 tx=0x7fd678005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:26.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.645+0000 7fd684ff9700 1 -- 172.21.15.93:0/959453332 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd67c0955c0 con 0x7fd6880ffe00 2023-12-14T18:25:26.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.675+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f58f81074e0 con 0x7f58f810cbc0 2023-12-14T18:25:26.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.675+0000 7f58f5ffb700 1 -- 172.21.15.93:0/1550478266 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f58f00998d0 con 0x7f58f810cbc0 2023-12-14T18:25:26.677 INFO:teuthology.orchestra.run.smithi093.stdout:55834574903 2023-12-14T18:25:26.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.677+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58e80a11e0 msgr2=0x7f58e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.679 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.677+0000 7f58fee89700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58e80a11e0 0x7f58e80a3690 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00b5c0 tx=0x7f58ec005fd0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f810cbc0 msgr2=0x7f58f819ce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f810cbc0 0x7f58f819ce30 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f58f0007d70 tx=0x7f58f000a340 comp rx=0 tx=0).stop 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 shutdown_connections 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58e80a11e0 0x7f58e80a3690 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f58f810cbc0 0x7f58f819ce30 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f58f8072e10 0x7f58f8198480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 --2- 172.21.15.93:0/1550478266 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f58f8073720 0x7f58f81989c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 >> 172.21.15.93:0/1550478266 conn(0x7f58f80fbde0 msgr2=0x7f58f8104620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 shutdown_connections 2023-12-14T18:25:26.680 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.678+0000 7f58fee89700 1 -- 172.21.15.93:0/1550478266 wait complete. 2023-12-14T18:25:26.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.685+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb6b8005190 con 0x7fb6cc100540 2023-12-14T18:25:26.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.686+0000 7fb6b3fff700 1 -- 172.21.15.93:0/2928677168 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb6bc0970a0 con 0x7fb6cc100540 2023-12-14T18:25:26.688 INFO:teuthology.orchestra.run.smithi093.stdout:133143986220 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb6b40a12e0 msgr2=0x7fb6b40a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb6b40a12e0 0x7fb6b40a3790 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fb6cc197a40 tx=0x7fb6c000a300 comp rx=0 tx=0).stop 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6cc100540 msgr2=0x7fb6cc196420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6cc100540 0x7fb6cc196420 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fb6bc00b580 tx=0x7fb6bc00f690 comp rx=0 tx=0).stop 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 shutdown_connections 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb6b40a12e0 0x7fb6b40a3790 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb6cc1055e0 0x7fb6cc19add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb6cc100540 0x7fb6cc196420 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 --2- 172.21.15.93:0/2928677168 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb6cc100ee0 0x7fb6cc196960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 >> 172.21.15.93:0/2928677168 conn(0x7fb6cc0fbdc0 msgr2=0x7fb6cc103e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 shutdown_connections 2023-12-14T18:25:26.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.688+0000 7fb6d14c7700 1 -- 172.21.15.93:0/2928677168 wait complete. 2023-12-14T18:25:26.799 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:26.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.849+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) v1 -- 0x7fd688066ea0 con 0x7fd6880ffe00 2023-12-14T18:25:26.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.849+0000 7fd684ff9700 1 -- 172.21.15.93:0/959453332 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fd67c098be0 con 0x7fd6880ffe00 2023-12-14T18:25:26.851 INFO:teuthology.orchestra.run.smithi093.stdout:236223201311 2023-12-14T18:25:26.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd6700a1260 msgr2=0x7fd6700a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd6700a1260 0x7fd6700a3710 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fd678000c00 tx=0x7fd678005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ffe00 msgr2=0x7fd688196a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ffe00 0x7fd688196a20 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fd67c00c8f0 tx=0x7fd67c00ccb0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 shutdown_connections 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd6700a1260 0x7fd6700a3710 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd688104500 0x7fd68819ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd6880ff460 0x7fd6881964e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 --2- 172.21.15.93:0/959453332 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd6880ffe00 0x7fd688196a20 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 >> 172.21.15.93:0/959453332 conn(0x7fd6880770a0 msgr2=0x7fd688068770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 shutdown_connections 2023-12-14T18:25:26.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:26.851+0000 7fd68db2e700 1 -- 172.21.15.93:0/959453332 wait complete. 2023-12-14T18:25:27.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:26 smithi093 ceph-mon[42070]: pgmap v249: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-12-14T18:25:27.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:26 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1550478266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-14T18:25:27.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:26 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2928677168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-14T18:25:27.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:26 smithi097 ceph-mon[43486]: pgmap v249: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-12-14T18:25:27.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:26 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1550478266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-14T18:25:27.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:26 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2928677168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-14T18:25:27.152 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986219 got 133143986220 for osd.4 2023-12-14T18:25:27.152 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:26 smithi160 ceph-mon[43840]: pgmap v249: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2023-12-14T18:25:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:26 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1550478266' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2023-12-14T18:25:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:26 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2928677168' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2023-12-14T18:25:27.315 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574901 got 55834574903 for osd.1 2023-12-14T18:25:27.315 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:27.952 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:28.002 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:28.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:27 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/959453332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-14T18:25:28.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:27 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/959453332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-14T18:25:28.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:27 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/959453332' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2023-12-14T18:25:28.521 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.519+0000 7f6136511700 1 -- 172.21.15.93:0/1394406898 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61301005b0 msgr2=0x7f6130100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.521 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.519+0000 7f6136511700 1 --2- 172.21.15.93:0/1394406898 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61301005b0 0x7f6130100980 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f6120009920 tx=0x7f6120009c30 comp rx=0 tx=0).stop 2023-12-14T18:25:28.521 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.519+0000 7f6136511700 1 -- 172.21.15.93:0/1394406898 shutdown_connections 2023-12-14T18:25:28.521 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.519+0000 7f6136511700 1 --2- 172.21.15.93:0/1394406898 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f61301005b0 0x7f6130100980 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.521 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.519+0000 7f6136511700 1 --2- 172.21.15.93:0/1394406898 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6130105650 0x7f613010d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.522 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.519+0000 7f6136511700 1 --2- 172.21.15.93:0/1394406898 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6130100f50 0x7f6130104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.522 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.519+0000 7f6136511700 1 -- 172.21.15.93:0/1394406898 >> 172.21.15.93:0/1394406898 conn(0x7f6130077110 msgr2=0x7f6130077510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:28.522 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.519+0000 7f6136511700 1 -- 172.21.15.93:0/1394406898 shutdown_connections 2023-12-14T18:25:28.522 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.519+0000 7f6136511700 1 -- 172.21.15.93:0/1394406898 wait complete. 2023-12-14T18:25:28.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f6136511700 1 Processor -- start 2023-12-14T18:25:28.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f6136511700 1 -- start start 2023-12-14T18:25:28.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f6136511700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61301005b0 0x7f61301964f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f6136511700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6130100f50 0x7f6130196a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f6136511700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6130105650 0x7f613019aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f6136511700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613019b4b0 con 0x7f6130100f50 2023-12-14T18:25:28.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f6136511700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613019b620 con 0x7f61301005b0 2023-12-14T18:25:28.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f6136511700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f613019b7e0 con 0x7f6130105650 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f612f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6130100f50 0x7f6130196a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f612f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6130100f50 0x7f6130196a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48754/0 (socket says 172.21.15.93:48754) 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f612f7fe700 1 -- 172.21.15.93:0/2507081657 learned_addr learned my addr 172.21.15.93:0/2507081657 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.520+0000 7f612ffff700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61301005b0 0x7f61301964f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f6134aae700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6130105650 0x7f613019aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f612f7fe700 1 -- 172.21.15.93:0/2507081657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6130105650 msgr2=0x7f613019aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f612f7fe700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6130105650 0x7f613019aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f612f7fe700 1 -- 172.21.15.93:0/2507081657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61301005b0 msgr2=0x7f61301964f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f612f7fe700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61301005b0 0x7f61301964f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f612f7fe700 1 -- 172.21.15.93:0/2507081657 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6120009520 con 0x7f6130100f50 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f6134aae700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6130105650 0x7f613019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:28.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f612f7fe700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6130100f50 0x7f6130196a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f612400e970 tx=0x7f612400ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:28.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f612d7fa700 1 -- 172.21.15.93:0/2507081657 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124009790 con 0x7f6130100f50 2023-12-14T18:25:28.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f612d7fa700 1 -- 172.21.15.93:0/2507081657 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f612400eec0 con 0x7f6130100f50 2023-12-14T18:25:28.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6130110390 con 0x7f6130100f50 2023-12-14T18:25:28.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f612d7fa700 1 -- 172.21.15.93:0/2507081657 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6124005590 con 0x7f6130100f50 2023-12-14T18:25:28.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.521+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6130110200 con 0x7f6130100f50 2023-12-14T18:25:28.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.522+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6130110520 con 0x7f6130100f50 2023-12-14T18:25:28.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.524+0000 7f612d7fa700 1 -- 172.21.15.93:0/2507081657 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f612400cbd0 con 0x7f6130100f50 2023-12-14T18:25:28.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.524+0000 7f612d7fa700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61180a11e0 0x7f61180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:28.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.524+0000 7f612d7fa700 1 -- 172.21.15.93:0/2507081657 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f6124014070 con 0x7f6130100f50 2023-12-14T18:25:28.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.524+0000 7f612ffff700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61180a11e0 0x7f61180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:28.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.525+0000 7f612ffff700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61180a11e0 0x7f61180a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f612000c010 tx=0x7f6120019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:28.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.528+0000 7f612d7fa700 1 -- 172.21.15.93:0/2507081657 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f61240964b0 con 0x7f6130100f50 2023-12-14T18:25:28.742 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.740+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f6130066ea0 con 0x7f6130100f50 2023-12-14T18:25:28.743 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.740+0000 7f612d7fa700 1 -- 172.21.15.93:0/2507081657 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f6124004ef0 con 0x7f6130100f50 2023-12-14T18:25:28.743 INFO:teuthology.orchestra.run.smithi093.stdout:154618822696 2023-12-14T18:25:28.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.742+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61180a11e0 msgr2=0x7f61180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.742+0000 7f6136511700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61180a11e0 0x7f61180a3690 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f612000c010 tx=0x7f6120019040 comp rx=0 tx=0).stop 2023-12-14T18:25:28.744 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.742+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6130100f50 msgr2=0x7f6130196a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:28.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.742+0000 7f6136511700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6130100f50 0x7f6130196a30 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f612400e970 tx=0x7f612400ec80 comp rx=0 tx=0).stop 2023-12-14T18:25:28.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.743+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 shutdown_connections 2023-12-14T18:25:28.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.743+0000 7f6136511700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f61180a11e0 0x7f61180a3690 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.743+0000 7f6136511700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6130105650 0x7f613019aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.743+0000 7f6136511700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f61301005b0 0x7f61301964f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.743+0000 7f6136511700 1 --2- 172.21.15.93:0/2507081657 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6130100f50 0x7f6130196a30 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:28.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.743+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 >> 172.21.15.93:0/2507081657 conn(0x7f6130077110 msgr2=0x7f6130103f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:28.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.743+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 shutdown_connections 2023-12-14T18:25:28.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:28.743+0000 7f6136511700 1 -- 172.21.15.93:0/2507081657 wait complete. 2023-12-14T18:25:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:29.060 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:28 smithi093 ceph-mon[42070]: pgmap v250: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-12-14T18:25:29.060 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:28 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2507081657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-14T18:25:29.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:28 smithi097 ceph-mon[43486]: pgmap v250: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-12-14T18:25:29.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:28 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2507081657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-14T18:25:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:28 smithi160 ceph-mon[43840]: pgmap v250: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 7 op/s 2023-12-14T18:25:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:28 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2507081657' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2023-12-14T18:25:29.504 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:29.611 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822696 got 154618822696 for osd.5 2023-12-14T18:25:29.612 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:29.816 INFO:tasks.cephadm.ceph_manager.ceph:need seq 236223201309 got 236223201311 for osd.8 2023-12-14T18:25:29.816 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:29.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 -- 172.21.15.93:0/1532266205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 msgr2=0x7f004410c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:29.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 --2- 172.21.15.93:0/1532266205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 0x7f004410c7b0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f0038009990 tx=0x7f0038009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 -- 172.21.15.93:0/1532266205 shutdown_connections 2023-12-14T18:25:29.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 --2- 172.21.15.93:0/1532266205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f004410ccf0 0x7f004410f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 --2- 172.21.15.93:0/1532266205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f00440ff140 0x7f00440ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 --2- 172.21.15.93:0/1532266205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 0x7f004410c7b0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 -- 172.21.15.93:0/1532266205 >> 172.21.15.93:0/1532266205 conn(0x7f00440770d0 msgr2=0x7f00440774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:29.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 -- 172.21.15.93:0/1532266205 shutdown_connections 2023-12-14T18:25:29.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 -- 172.21.15.93:0/1532266205 wait complete. 2023-12-14T18:25:29.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.990+0000 7f00497c9700 1 Processor -- start 2023-12-14T18:25:29.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00497c9700 1 -- start start 2023-12-14T18:25:29.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00497c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00440ff140 0x7f0044198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:29.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00497c9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 0x7f0044198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00497c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004410ccf0 0x7f004419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00497c9700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004419d550 con 0x7f00440ffa50 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00497c9700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004419d6c0 con 0x7f004410ccf0 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00497c9700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f004419d880 con 0x7f00440ff140 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 0x7f0044198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00427fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 0x7f0044198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48774/0 (socket says 172.21.15.93:48774) 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00427fc700 1 -- 172.21.15.93:0/2637858487 learned_addr learned my addr 172.21.15.93:0/2637858487 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00437fe700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004410ccf0 0x7f004419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f0042ffd700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00440ff140 0x7f0044198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00427fc700 1 -- 172.21.15.93:0/2637858487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00440ff140 msgr2=0x7f0044198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00427fc700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00440ff140 0x7f0044198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00427fc700 1 -- 172.21.15.93:0/2637858487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004410ccf0 msgr2=0x7f004419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00427fc700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004410ccf0 0x7f004419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00427fc700 1 -- 172.21.15.93:0/2637858487 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00380095f0 con 0x7f00440ffa50 2023-12-14T18:25:29.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00437fe700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004410ccf0 0x7f004419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:29.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.991+0000 7f00427fc700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 0x7f0044198ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00380098e0 tx=0x7f003800f720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:29.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.992+0000 7f002bfff700 1 -- 172.21.15.93:0/2637858487 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f003801d070 con 0x7f00440ffa50 2023-12-14T18:25:29.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.992+0000 7f00497c9700 1 -- 172.21.15.93:0/2637858487 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0044112550 con 0x7f00440ffa50 2023-12-14T18:25:29.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.992+0000 7f002bfff700 1 -- 172.21.15.93:0/2637858487 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0038005510 con 0x7f00440ffa50 2023-12-14T18:25:29.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.992+0000 7f002bfff700 1 -- 172.21.15.93:0/2637858487 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0038004c90 con 0x7f00440ffa50 2023-12-14T18:25:29.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.992+0000 7f00497c9700 1 -- 172.21.15.93:0/2637858487 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00441123c0 con 0x7f00440ffa50 2023-12-14T18:25:29.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.993+0000 7f0029ffb700 1 -- 172.21.15.93:0/2637858487 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00441126e0 con 0x7f00440ffa50 2023-12-14T18:25:29.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.994+0000 7f002bfff700 1 -- 172.21.15.93:0/2637858487 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f0038021410 con 0x7f00440ffa50 2023-12-14T18:25:29.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.994+0000 7f002bfff700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f002c0a11e0 0x7f002c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:29.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.995+0000 7f0042ffd700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f002c0a11e0 0x7f002c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:29.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.995+0000 7f002bfff700 1 -- 172.21.15.93:0/2637858487 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f00380cdb80 con 0x7f00440ffa50 2023-12-14T18:25:29.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:29.995+0000 7f0042ffd700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f002c0a11e0 0x7f002c0a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0034005950 tx=0x7f00340058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:30.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.003+0000 7f002bfff700 1 -- 172.21.15.93:0/2637858487 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f003805b730 con 0x7f00440ffa50 2023-12-14T18:25:30.049 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.055+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3098982800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec100370 msgr2=0x7fa6ec10c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.055+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3098982800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec100370 0x7fa6ec10c000 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fa6e400b0d0 tx=0x7fa6e400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.055+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3098982800 shutdown_connections 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.055+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3098982800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec100370 0x7fa6ec10c000 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.055+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3098982800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec104180 0x7fa6ec104550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.055+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3098982800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec0ff9c0 0x7fa6ec0ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.055+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3098982800 >> 172.21.15.93:0/3098982800 conn(0x7fa6ec0758a0 msgr2=0x7fa6ec075ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.055+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3098982800 shutdown_connections 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.055+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3098982800 wait complete. 2023-12-14T18:25:30.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.056+0000 7fa6f1d7c700 1 Processor -- start 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.056+0000 7fa6f1d7c700 1 -- start start 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6f1d7c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec0ff9c0 0x7fa6ec10bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6f1d7c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec100370 0x7fa6ec107d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6f1d7c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec104180 0x7fa6ec108240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6f1d7c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec108890 con 0x7fa6ec104180 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6f1d7c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec108a00 con 0x7fa6ec100370 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6f1d7c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6ec108bc0 con 0x7fa6ec0ff9c0 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec104180 0x7fa6ec108240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6ebfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec104180 0x7fa6ec108240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:48782/0 (socket says 172.21.15.93:48782) 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6ebfff700 1 -- 172.21.15.93:0/3253036136 learned_addr learned my addr 172.21.15.93:0/3253036136 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6eaffd700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec100370 0x7fa6ec107d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6eb7fe700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec0ff9c0 0x7fa6ec10bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6ebfff700 1 -- 172.21.15.93:0/3253036136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec0ff9c0 msgr2=0x7fa6ec10bca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6ebfff700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec0ff9c0 0x7fa6ec10bca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.068 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6ebfff700 1 -- 172.21.15.93:0/3253036136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec100370 msgr2=0x7fa6ec107d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6ebfff700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec100370 0x7fa6ec107d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6ebfff700 1 -- 172.21.15.93:0/3253036136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6e4009d00 con 0x7fa6ec104180 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6eaffd700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec100370 0x7fa6ec107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.057+0000 7fa6ebfff700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec104180 0x7fa6ec108240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007870 tx=0x7fa6e40078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.058+0000 7fa6e8ff9700 1 -- 172.21.15.93:0/3253036136 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e400e040 con 0x7fa6ec104180 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.058+0000 7fa6e8ff9700 1 -- 172.21.15.93:0/3253036136 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa6e4005070 con 0x7fa6ec104180 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.058+0000 7fa6e8ff9700 1 -- 172.21.15.93:0/3253036136 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6e401b6a0 con 0x7fa6ec104180 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.059+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6ec10d630 con 0x7fa6ec104180 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.059+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6ec10d4a0 con 0x7fa6ec104180 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.060+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6ec10d7c0 con 0x7fa6ec104180 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.062+0000 7fa6e8ff9700 1 -- 172.21.15.93:0/3253036136 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa6e40051e0 con 0x7fa6ec104180 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.063+0000 7fa6e8ff9700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa6d40a12e0 0x7fa6d40a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.063+0000 7fa6e8ff9700 1 -- 172.21.15.93:0/3253036136 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa6e40cd750 con 0x7fa6ec104180 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.064+0000 7fa6eb7fe700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa6d40a12e0 0x7fa6d40a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:30.069 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.064+0000 7fa6eb7fe700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa6d40a12e0 0x7fa6d40a3790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005950 tx=0x7fa6dc0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:30.070 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.068+0000 7fa6e8ff9700 1 -- 172.21.15.93:0/3253036136 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa6e4096b80 con 0x7fa6ec104180 2023-12-14T18:25:30.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.221+0000 7f0029ffb700 1 -- 172.21.15.93:0/2637858487 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0044066ea0 con 0x7f00440ffa50 2023-12-14T18:25:30.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.222+0000 7f002bfff700 1 -- 172.21.15.93:0/2637858487 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f0038096fb0 con 0x7f00440ffa50 2023-12-14T18:25:30.223 INFO:teuthology.orchestra.run.smithi093.stdout:206158430242 2023-12-14T18:25:30.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.224+0000 7f0029ffb700 1 -- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f002c0a11e0 msgr2=0x7f002c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:30.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.224+0000 7f0029ffb700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f002c0a11e0 0x7f002c0a3690 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0034005950 tx=0x7f00340058e0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.224+0000 7f0029ffb700 1 -- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 msgr2=0x7f0044198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:30.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.224+0000 7f0029ffb700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 0x7f0044198ad0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f00380098e0 tx=0x7f003800f720 comp rx=0 tx=0).stop 2023-12-14T18:25:30.226 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.225+0000 7f0029ffb700 1 -- 172.21.15.93:0/2637858487 shutdown_connections 2023-12-14T18:25:30.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.225+0000 7f0029ffb700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f002c0a11e0 0x7f002c0a3690 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.225+0000 7f0029ffb700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f004410ccf0 0x7f004419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.225+0000 7f0029ffb700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f00440ff140 0x7f0044198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.225+0000 7f0029ffb700 1 --2- 172.21.15.93:0/2637858487 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f00440ffa50 0x7f0044198ad0 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.225+0000 7f0029ffb700 1 -- 172.21.15.93:0/2637858487 >> 172.21.15.93:0/2637858487 conn(0x7f00440770d0 msgr2=0x7f004410dfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:30.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.225+0000 7f0029ffb700 1 -- 172.21.15.93:0/2637858487 shutdown_connections 2023-12-14T18:25:30.227 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.225+0000 7f0029ffb700 1 -- 172.21.15.93:0/2637858487 wait complete. 2023-12-14T18:25:30.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.275+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) v1 -- 0x7fa6ec066ea0 con 0x7fa6ec104180 2023-12-14T18:25:30.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.275+0000 7fa6e8ff9700 1 -- 172.21.15.93:0/3253036136 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7fa6e409a1a0 con 0x7fa6ec104180 2023-12-14T18:25:30.277 INFO:teuthology.orchestra.run.smithi093.stdout:322122547221 2023-12-14T18:25:30.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa6d40a12e0 msgr2=0x7fa6d40a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa6d40a12e0 0x7fa6d40a3790 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc005950 tx=0x7fa6dc0058e0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec104180 msgr2=0x7fa6ec108240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec104180 0x7fa6ec108240 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa6e4007870 tx=0x7fa6e40078a0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 shutdown_connections 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa6d40a12e0 0x7fa6d40a3790 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6ec104180 0x7fa6ec108240 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa6ec0ff9c0 0x7fa6ec10bca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 --2- 172.21.15.93:0/3253036136 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6ec100370 0x7fa6ec107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 >> 172.21.15.93:0/3253036136 conn(0x7fa6ec0758a0 msgr2=0x7fa6ec0fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 shutdown_connections 2023-12-14T18:25:30.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:30.278+0000 7fa6f1d7c700 1 -- 172.21.15.93:0/3253036136 wait complete. 2023-12-14T18:25:30.487 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:30.888 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:31.120 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:31 smithi093 ceph-mon[42070]: pgmap v251: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:25:31.120 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:31 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2637858487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-14T18:25:31.120 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:31 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3253036136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-14T18:25:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:31 smithi097 ceph-mon[43486]: pgmap v251: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:25:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:31 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2637858487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-14T18:25:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:31 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3253036136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-14T18:25:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:31 smithi160 ceph-mon[43840]: pgmap v251: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:25:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:31 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2637858487' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2023-12-14T18:25:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:31 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3253036136' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2023-12-14T18:25:31.499 INFO:tasks.cephadm.ceph_manager.ceph:need seq 322122547219 got 322122547221 for osd.11 2023-12-14T18:25:31.500 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:31.721 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430240 got 206158430242 for osd.7 2023-12-14T18:25:31.721 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:32.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 -- 172.21.15.93:0/1494614446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 msgr2=0x7f4ce40ff830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 --2- 172.21.15.93:0/1494614446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 0x7f4ce40ff830 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f4cd40099c0 tx=0x7f4cd4009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 -- 172.21.15.93:0/1494614446 shutdown_connections 2023-12-14T18:25:32.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 --2- 172.21.15.93:0/1494614446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 0x7f4ce40ff830 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 --2- 172.21.15.93:0/1494614446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce4104500 0x7f4ce410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 --2- 172.21.15.93:0/1494614446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce40ffe00 0x7f4ce4103dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 -- 172.21.15.93:0/1494614446 >> 172.21.15.93:0/1494614446 conn(0x7f4ce40770a0 msgr2=0x7f4ce40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 -- 172.21.15.93:0/1494614446 shutdown_connections 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 -- 172.21.15.93:0/1494614446 wait complete. 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.885+0000 7f4cea467700 1 Processor -- start 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4cea467700 1 -- start start 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4cea467700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 0x7f4ce41952c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4cea467700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce40ffe00 0x7f4ce4195800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4cea467700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce4104500 0x7f4ce4199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4cea467700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce419a280 con 0x7f4ce40ff460 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4cea467700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce419a3f0 con 0x7f4ce40ffe00 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4cea467700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ce419a5b0 con 0x7f4ce4104500 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 0x7f4ce41952c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 0x7f4ce41952c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47538/0 (socket says 172.21.15.93:47538) 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce3fff700 1 -- 172.21.15.93:0/962987498 learned_addr learned my addr 172.21.15.93:0/962987498 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce37fe700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce40ffe00 0x7f4ce4195800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce8a04700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce4104500 0x7f4ce4199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce3fff700 1 -- 172.21.15.93:0/962987498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce4104500 msgr2=0x7f4ce4199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce3fff700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce4104500 0x7f4ce4199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce3fff700 1 -- 172.21.15.93:0/962987498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce40ffe00 msgr2=0x7f4ce4195800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce3fff700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce40ffe00 0x7f4ce4195800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce3fff700 1 -- 172.21.15.93:0/962987498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4cd40095f0 con 0x7f4ce40ff460 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.886+0000 7f4ce37fe700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce40ffe00 0x7f4ce4195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.887+0000 7f4ce3fff700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 0x7f4ce41952c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009990 tx=0x7f4cd4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.887+0000 7f4ce17fa700 1 -- 172.21.15.93:0/962987498 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd401d070 con 0x7f4ce40ff460 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.887+0000 7f4ce17fa700 1 -- 172.21.15.93:0/962987498 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4cd4004010 con 0x7f4ce40ff460 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.887+0000 7f4ce17fa700 1 -- 172.21.15.93:0/962987498 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4022d40 con 0x7f4ce40ff460 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.887+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ce410f240 con 0x7f4ce40ff460 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.887+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ce410f0b0 con 0x7f4ce40ff460 2023-12-14T18:25:32.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.888+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ce410f3d0 con 0x7f4ce40ff460 2023-12-14T18:25:32.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.889+0000 7f4ce17fa700 1 -- 172.21.15.93:0/962987498 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4cd4022470 con 0x7f4ce40ff460 2023-12-14T18:25:32.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.889+0000 7f4ce17fa700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ccc0a12e0 0x7f4ccc0a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.889+0000 7f4ce37fe700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ccc0a12e0 0x7f4ccc0a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.889+0000 7f4ce17fa700 1 -- 172.21.15.93:0/962987498 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4cd40ce3f0 con 0x7f4ce40ff460 2023-12-14T18:25:32.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.890+0000 7f4ce37fe700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ccc0a12e0 0x7f4ccc0a3790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ce41968e0 tx=0x7f4cd800b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:32.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:31.893+0000 7f4ce17fa700 1 -- 172.21.15.93:0/962987498 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4cd4097820 con 0x7f4ce40ff460 2023-12-14T18:25:32.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.092+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) v1 -- 0x7f4ce41a2340 con 0x7f4ce40ff460 2023-12-14T18:25:32.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.093+0000 7f4ce17fa700 1 -- 172.21.15.93:0/962987498 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4cd409ae40 con 0x7f4ce40ff460 2023-12-14T18:25:32.095 INFO:teuthology.orchestra.run.smithi093.stdout:266287972379 2023-12-14T18:25:32.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ccc0a12e0 msgr2=0x7f4ccc0a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ccc0a12e0 0x7f4ccc0a3790 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f4ce41968e0 tx=0x7f4cd800b5f0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 msgr2=0x7f4ce41952c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 0x7f4ce41952c0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4009990 tx=0x7f4cd4009580 comp rx=0 tx=0).stop 2023-12-14T18:25:32.097 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 shutdown_connections 2023-12-14T18:25:32.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ccc0a12e0 0x7f4ccc0a3790 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ce40ff460 0x7f4ce41952c0 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ce4104500 0x7f4ce4199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 --2- 172.21.15.93:0/962987498 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ce40ffe00 0x7f4ce4195800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 >> 172.21.15.93:0/962987498 conn(0x7f4ce40770a0 msgr2=0x7f4ce40fccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:32.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 shutdown_connections 2023-12-14T18:25:32.098 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.095+0000 7f4cea467700 1 -- 172.21.15.93:0/962987498 wait complete. 2023-12-14T18:25:32.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.098+0000 7fa604ba6700 1 -- 172.21.15.93:0/545272450 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 msgr2=0x7fa6000ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.098+0000 7fa604ba6700 1 --2- 172.21.15.93:0/545272450 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 0x7fa6000ff840 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fa5f00099c0 tx=0x7fa5f0009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.098+0000 7fa604ba6700 1 -- 172.21.15.93:0/545272450 shutdown_connections 2023-12-14T18:25:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.098+0000 7fa604ba6700 1 --2- 172.21.15.93:0/545272450 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 0x7fa6000ff840 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.098+0000 7fa604ba6700 1 --2- 172.21.15.93:0/545272450 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600104510 0x7fa60010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.098+0000 7fa604ba6700 1 --2- 172.21.15.93:0/545272450 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6000ffe10 0x7fa600103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.098+0000 7fa604ba6700 1 -- 172.21.15.93:0/545272450 >> 172.21.15.93:0/545272450 conn(0x7fa6000770d0 msgr2=0x7fa6000774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.098+0000 7fa604ba6700 1 -- 172.21.15.93:0/545272450 shutdown_connections 2023-12-14T18:25:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.098+0000 7fa604ba6700 1 -- 172.21.15.93:0/545272450 wait complete. 2023-12-14T18:25:32.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa604ba6700 1 Processor -- start 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa604ba6700 1 -- start start 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa604ba6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 0x7fa600195210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa604ba6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6000ffe10 0x7fa600195750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa604ba6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600104510 0x7fa600199bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa604ba6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60019a1d0 con 0x7fa6000ff470 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa604ba6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60019a340 con 0x7fa6000ffe10 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa604ba6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa60019a500 con 0x7fa600104510 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 0x7fa600195210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 0x7fa600195210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47560/0 (socket says 172.21.15.93:47560) 2023-12-14T18:25:32.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fe59c700 1 -- 172.21.15.93:0/3769079702 learned_addr learned my addr 172.21.15.93:0/3769079702 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fdd9b700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6000ffe10 0x7fa600195750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fed9d700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600104510 0x7fa600199bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fe59c700 1 -- 172.21.15.93:0/3769079702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600104510 msgr2=0x7fa600199bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fe59c700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600104510 0x7fa600199bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fe59c700 1 -- 172.21.15.93:0/3769079702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6000ffe10 msgr2=0x7fa600195750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fe59c700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6000ffe10 0x7fa600195750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.099+0000 7fa5fe59c700 1 -- 172.21.15.93:0/3769079702 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f00095f0 con 0x7fa6000ff470 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.100+0000 7fa5fe59c700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 0x7fa600195210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b580 tx=0x7fa5f000f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.100+0000 7fa5ef7fe700 1 -- 172.21.15.93:0/3769079702 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f001d070 con 0x7fa6000ff470 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.100+0000 7fa5ef7fe700 1 -- 172.21.15.93:0/3769079702 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa5f00054f0 con 0x7fa6000ff470 2023-12-14T18:25:32.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.100+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa60010f250 con 0x7fa6000ff470 2023-12-14T18:25:32.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.100+0000 7fa5ef7fe700 1 -- 172.21.15.93:0/3769079702 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5f0004b30 con 0x7fa6000ff470 2023-12-14T18:25:32.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.100+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa60010f0c0 con 0x7fa6000ff470 2023-12-14T18:25:32.529 INFO:teuthology.orchestra.run.smithi093.stdout:103079215153 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.101+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa60010f3e0 con 0x7fa6000ff470 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.102+0000 7fa5ef7fe700 1 -- 172.21.15.93:0/3769079702 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa5f000fb80 con 0x7fa6000ff470 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.102+0000 7fa5ef7fe700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa5e80a11e0 0x7fa5e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.103+0000 7fa5fdd9b700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa5e80a11e0 0x7fa5e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.103+0000 7fa5ef7fe700 1 -- 172.21.15.93:0/3769079702 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fa5f00cda40 con 0x7fa6000ff470 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.103+0000 7fa5fdd9b700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa5e80a11e0 0x7fa5e80a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa600196830 tx=0x7fa5f400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.107+0000 7fa5ef7fe700 1 -- 172.21.15.93:0/3769079702 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa5f0096ed0 con 0x7fa6000ff470 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.308+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fa600196610 con 0x7fa6000ff470 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.308+0000 7fa5ef7fe700 1 -- 172.21.15.93:0/3769079702 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fa5f0026090 con 0x7fa6000ff470 2023-12-14T18:25:32.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.310+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa5e80a11e0 msgr2=0x7fa5e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.310+0000 7fa604ba6700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa5e80a11e0 0x7fa5e80a3690 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fa600196830 tx=0x7fa5f400a380 comp rx=0 tx=0).stop 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.310+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 msgr2=0x7fa600195210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.310+0000 7fa604ba6700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 0x7fa600195210 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fa5f000b580 tx=0x7fa5f000f690 comp rx=0 tx=0).stop 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.311+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 shutdown_connections 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.311+0000 7fa604ba6700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa5e80a11e0 0x7fa5e80a3690 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.311+0000 7fa604ba6700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa6000ff470 0x7fa600195210 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.311+0000 7fa604ba6700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa600104510 0x7fa600199bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.311+0000 7fa604ba6700 1 --2- 172.21.15.93:0/3769079702 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa6000ffe10 0x7fa600195750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.311+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 >> 172.21.15.93:0/3769079702 conn(0x7fa6000770d0 msgr2=0x7fa6000fcc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.311+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 shutdown_connections 2023-12-14T18:25:32.531 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.311+0000 7fa604ba6700 1 -- 172.21.15.93:0/3769079702 wait complete. 2023-12-14T18:25:32.783 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.607+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/2309662570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e0101cf0 msgr2=0x7fd0e010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.607+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/2309662570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e0101cf0 0x7fd0e010a3c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fd0c8009990 tx=0x7fd0c8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.607+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/2309662570 shutdown_connections 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.607+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/2309662570 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e010a900 0x7fd0e010cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.607+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/2309662570 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0e01013e0 0x7fd0e01017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.607+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/2309662570 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e0101cf0 0x7fd0e010a3c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.607+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/2309662570 >> 172.21.15.93:0/2309662570 conn(0x7fd0e00770d0 msgr2=0x7fd0e00774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.607+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/2309662570 shutdown_connections 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.607+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/2309662570 wait complete. 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0e4ddf700 1 Processor -- start 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0e4ddf700 1 -- start start 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0e4ddf700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e01013e0 0x7fd0e0196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0e4ddf700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0e0101cf0 0x7fd0e01969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0e4ddf700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e010a900 0x7fd0e019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.784 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0e4ddf700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e019b420 con 0x7fd0e010a900 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0e4ddf700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e019b590 con 0x7fd0e01013e0 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0e4ddf700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0e019b750 con 0x7fd0e0101cf0 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ded9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e010a900 0x7fd0e019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ded9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e010a900 0x7fd0e019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47582/0 (socket says 172.21.15.93:47582) 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ded9d700 1 -- 172.21.15.93:0/3984543911 learned_addr learned my addr 172.21.15.93:0/3984543911 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ddd9b700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0e0101cf0 0x7fd0e01969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0de59c700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e01013e0 0x7fd0e0196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ded9d700 1 -- 172.21.15.93:0/3984543911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0e0101cf0 msgr2=0x7fd0e01969a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ded9d700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0e0101cf0 0x7fd0e01969a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ded9d700 1 -- 172.21.15.93:0/3984543911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e01013e0 msgr2=0x7fd0e0196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ded9d700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e01013e0 0x7fd0e0196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ded9d700 1 -- 172.21.15.93:0/3984543911 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0c80095f0 con 0x7fd0e010a900 2023-12-14T18:25:32.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0de59c700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e01013e0 0x7fd0e0196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.608+0000 7fd0ded9d700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e010a900 0x7fd0e019ae10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000ada0 tx=0x7fd0d000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.609+0000 7fd0d77fe700 1 -- 172.21.15.93:0/3984543911 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d000f070 con 0x7fd0e010a900 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.609+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0e0110380 con 0x7fd0e010a900 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.609+0000 7fd0d77fe700 1 -- 172.21.15.93:0/3984543911 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fd0d0007ad0 con 0x7fd0e010a900 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.609+0000 7fd0d77fe700 1 -- 172.21.15.93:0/3984543911 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0d0005310 con 0x7fd0e010a900 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.609+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0e01101f0 con 0x7fd0e010a900 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.610+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0e0110510 con 0x7fd0e010a900 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.611+0000 7fd0d77fe700 1 -- 172.21.15.93:0/3984543911 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fd0d000c980 con 0x7fd0e010a900 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.611+0000 7fd0d77fe700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0cc0a11e0 0x7fd0cc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.611+0000 7fd0d77fe700 1 -- 172.21.15.93:0/3984543911 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7fd0d00cc760 con 0x7fd0e010a900 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.611+0000 7fd0de59c700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0cc0a11e0 0x7fd0cc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.612+0000 7fd0de59c700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0cc0a11e0 0x7fd0cc0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8005950 tx=0x7fd0d80058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:32.786 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.615+0000 7fd0d77fe700 1 -- 172.21.15.93:0/3984543911 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fd0d0095bf0 con 0x7fd0e010a900 2023-12-14T18:25:32.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.812+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd0e01a34e0 con 0x7fd0e010a900 2023-12-14T18:25:32.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.812+0000 7fd0d77fe700 1 -- 172.21.15.93:0/3984543911 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd0d0099210 con 0x7fd0e010a900 2023-12-14T18:25:32.814 INFO:teuthology.orchestra.run.smithi093.stdout:77309411380 2023-12-14T18:25:32.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0cc0a11e0 msgr2=0x7fd0cc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0cc0a11e0 0x7fd0cc0a3690 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fd0d8005950 tx=0x7fd0d80058e0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e010a900 msgr2=0x7fd0e019ae10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e010a900 0x7fd0e019ae10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fd0d000ada0 tx=0x7fd0d000c330 comp rx=0 tx=0).stop 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 shutdown_connections 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fd0cc0a11e0 0x7fd0cc0a3690 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fd0e010a900 0x7fd0e019ae10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fd0e01013e0 0x7fd0e0196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 --2- 172.21.15.93:0/3984543911 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fd0e0101cf0 0x7fd0e01969a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.814+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 >> 172.21.15.93:0/3984543911 conn(0x7fd0e00770d0 msgr2=0x7fd0e00fce00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.815+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 shutdown_connections 2023-12-14T18:25:32.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:32.815+0000 7fd0e4ddf700 1 -- 172.21.15.93:0/3984543911 wait complete. 2023-12-14T18:25:33.111 INFO:tasks.cephadm.ceph_manager.ceph:need seq 266287972377 got 266287972379 for osd.9 2023-12-14T18:25:33.111 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:33.292 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:33 smithi093 ceph-mon[42070]: pgmap v252: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:25:33.292 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/962987498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-14T18:25:33.292 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3769079702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-14T18:25:33.292 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3984543911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-14T18:25:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:33 smithi097 ceph-mon[43486]: pgmap v252: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:25:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/962987498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-14T18:25:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3769079702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-14T18:25:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3984543911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-14T18:25:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:33 smithi160 ceph-mon[43840]: pgmap v252: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:25:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/962987498' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2023-12-14T18:25:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3769079702' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2023-12-14T18:25:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3984543911' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2023-12-14T18:25:33.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.505+0000 7f7caa1b4700 1 -- 172.21.15.93:0/3606428832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100540 msgr2=0x7f7ca4100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:33.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.505+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/3606428832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100540 0x7f7ca4100910 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f7c940099c0 tx=0x7f7c94009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.505+0000 7f7caa1b4700 1 -- 172.21.15.93:0/3606428832 shutdown_connections 2023-12-14T18:25:33.507 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.505+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/3606428832 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100540 0x7f7ca4100910 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.505+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/3606428832 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca41055e0 0x7f7ca410d160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.505+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/3606428832 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ca4100ee0 0x7f7ca4104eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.505+0000 7f7caa1b4700 1 -- 172.21.15.93:0/3606428832 >> 172.21.15.93:0/3606428832 conn(0x7f7ca40fbdc0 msgr2=0x7f7ca40fe1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:33.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.505+0000 7f7caa1b4700 1 -- 172.21.15.93:0/3606428832 shutdown_connections 2023-12-14T18:25:33.508 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.505+0000 7f7caa1b4700 1 -- 172.21.15.93:0/3606428832 wait complete. 2023-12-14T18:25:33.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.506+0000 7f7caa1b4700 1 Processor -- start 2023-12-14T18:25:33.510 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.506+0000 7f7caa1b4700 1 -- start start 2023-12-14T18:25:33.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7caa1b4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca4100540 0x7f7ca4196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:33.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7caa1b4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100ee0 0x7f7ca4196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:33.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7caa1b4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ca41055e0 0x7f7ca419add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:33.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7caa1b4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca419b3e0 con 0x7f7ca4100ee0 2023-12-14T18:25:33.511 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7caa1b4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca419b550 con 0x7f7ca4100540 2023-12-14T18:25:33.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7caa1b4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ca419b710 con 0x7f7ca41055e0 2023-12-14T18:25:33.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100ee0 0x7f7ca4196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:33.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100ee0 0x7f7ca4196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47598/0 (socket says 172.21.15.93:47598) 2023-12-14T18:25:33.512 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca2ffd700 1 -- 172.21.15.93:0/2469465404 learned_addr learned my addr 172.21.15.93:0/2469465404 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:33.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca37fe700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca4100540 0x7f7ca4196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:33.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca3fff700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ca41055e0 0x7f7ca419add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:33.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca2ffd700 1 -- 172.21.15.93:0/2469465404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ca41055e0 msgr2=0x7f7ca419add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:33.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca2ffd700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ca41055e0 0x7f7ca419add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca2ffd700 1 -- 172.21.15.93:0/2469465404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca4100540 msgr2=0x7f7ca4196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:33.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca2ffd700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca4100540 0x7f7ca4196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca2ffd700 1 -- 172.21.15.93:0/2469465404 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c940095f0 con 0x7f7ca4100ee0 2023-12-14T18:25:33.513 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca37fe700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca4100540 0x7f7ca4196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:33.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca2ffd700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100ee0 0x7f7ca4196960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d980 tx=0x7f7c9800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:33.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca0ff9700 1 -- 172.21.15.93:0/2469465404 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c98009b40 con 0x7f7ca4100ee0 2023-12-14T18:25:33.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ca4110370 con 0x7f7ca4100ee0 2023-12-14T18:25:33.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca0ff9700 1 -- 172.21.15.93:0/2469465404 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c9800de60 con 0x7f7ca4100ee0 2023-12-14T18:25:33.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.507+0000 7f7ca0ff9700 1 -- 172.21.15.93:0/2469465404 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c9800f620 con 0x7f7ca4100ee0 2023-12-14T18:25:33.514 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.508+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ca41101e0 con 0x7f7ca4100ee0 2023-12-14T18:25:33.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.509+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ca4110500 con 0x7f7ca4100ee0 2023-12-14T18:25:33.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.510+0000 7f7ca0ff9700 1 -- 172.21.15.93:0/2469465404 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7c98010460 con 0x7f7ca4100ee0 2023-12-14T18:25:33.515 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.510+0000 7f7ca0ff9700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7c8c0a11e0 0x7f7c8c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:33.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.510+0000 7f7ca0ff9700 1 -- 172.21.15.93:0/2469465404 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f7c980cc1c0 con 0x7f7ca4100ee0 2023-12-14T18:25:33.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.510+0000 7f7ca37fe700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7c8c0a11e0 0x7f7c8c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:33.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.511+0000 7f7ca37fe700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7c8c0a11e0 0x7f7c8c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009fd0 tx=0x7f7c94005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:33.516 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.514+0000 7f7ca0ff9700 1 -- 172.21.15.93:0/2469465404 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7c980955c0 con 0x7f7ca4100ee0 2023-12-14T18:25:33.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.710+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f7ca41a3500 con 0x7f7ca4100ee0 2023-12-14T18:25:33.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.711+0000 7f7ca0ff9700 1 -- 172.21.15.93:0/2469465404 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f7c98098be0 con 0x7f7ca4100ee0 2023-12-14T18:25:33.713 INFO:teuthology.orchestra.run.smithi093.stdout:34359738428 2023-12-14T18:25:33.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7c8c0a11e0 msgr2=0x7f7c8c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:33.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7c8c0a11e0 0x7f7c8c0a3690 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f7c94009fd0 tx=0x7f7c94005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100ee0 msgr2=0x7f7ca4196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:33.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100ee0 0x7f7ca4196960 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f7c9800d980 tx=0x7f7c9800dd40 comp rx=0 tx=0).stop 2023-12-14T18:25:33.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 shutdown_connections 2023-12-14T18:25:33.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7c8c0a11e0 0x7f7c8c0a3690 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7ca41055e0 0x7f7ca419add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7ca4100540 0x7f7ca4196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 --2- 172.21.15.93:0/2469465404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7ca4100ee0 0x7f7ca4196960 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:33.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 >> 172.21.15.93:0/2469465404 conn(0x7f7ca40fbdc0 msgr2=0x7f7ca40fc800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:33.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 shutdown_connections 2023-12-14T18:25:33.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:33.713+0000 7f7caa1b4700 1 -- 172.21.15.93:0/2469465404 wait complete. 2023-12-14T18:25:33.742 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411378 got 77309411380 for osd.2 2023-12-14T18:25:33.742 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:33.960 INFO:tasks.cephadm.ceph_manager.ceph:need seq 103079215150 got 103079215153 for osd.3 2023-12-14T18:25:33.960 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:34.216 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738426 got 34359738428 for osd.0 2023-12-14T18:25:34.216 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:34.276 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:34 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2469465404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-14T18:25:34.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:34 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2469465404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-14T18:25:34.482 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.419+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3401442313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100490 msgr2=0x7efcd0100900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.419+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3401442313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100490 0x7efcd0100900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7efcc4009990 tx=0x7efcc4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.419+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3401442313 shutdown_connections 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.419+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3401442313 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd0100e40 0x7efcd010d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.419+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3401442313 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcd0106370 0x7efcd0106740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.419+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3401442313 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100490 0x7efcd0100900 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.419+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3401442313 >> 172.21.15.93:0/3401442313 conn(0x7efcd0078430 msgr2=0x7efcd0078830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.419+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3401442313 shutdown_connections 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.419+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3401442313 wait complete. 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.420+0000 7efcd5ee3700 1 Processor -- start 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.420+0000 7efcd5ee3700 1 -- start start 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.420+0000 7efcd5ee3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd0100490 0x7efcd0196460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:34.483 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.420+0000 7efcd5ee3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100e40 0x7efcd01969a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.420+0000 7efcd5ee3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcd0106370 0x7efcd019ae10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcd5ee3700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd019b420 con 0x7efcd0100e40 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcd5ee3700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd019b590 con 0x7efcd0100490 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcd5ee3700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcd019b750 con 0x7efcd0106370 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcceffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100e40 0x7efcd01969a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcceffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100e40 0x7efcd01969a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47622/0 (socket says 172.21.15.93:47622) 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcceffd700 1 -- 172.21.15.93:0/3811612100 learned_addr learned my addr 172.21.15.93:0/3811612100 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efccf7fe700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd0100490 0x7efcd0196460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcceffd700 1 -- 172.21.15.93:0/3811612100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcd0106370 msgr2=0x7efcd019ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efccffff700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcd0106370 0x7efcd019ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:34.484 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcceffd700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcd0106370 0x7efcd019ae10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcceffd700 1 -- 172.21.15.93:0/3811612100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd0100490 msgr2=0x7efcd0196460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcceffd700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd0100490 0x7efcd0196460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcceffd700 1 -- 172.21.15.93:0/3811612100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcc40095f0 con 0x7efcd0100e40 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcceffd700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100e40 0x7efcd01969a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efcc4006010 tx=0x7efcc400f7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcccff9700 1 -- 172.21.15.93:0/3811612100 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc401d070 con 0x7efcd0100e40 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcccff9700 1 -- 172.21.15.93:0/3811612100 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7efcc400fc60 con 0x7efcd0100e40 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcd0110400 con 0x7efcd0100e40 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcccff9700 1 -- 172.21.15.93:0/3811612100 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcc4017720 con 0x7efcd0100e40 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.421+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcd0110270 con 0x7efcd0100e40 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.422+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcd0110590 con 0x7efcd0100e40 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.423+0000 7efcccff9700 1 -- 172.21.15.93:0/3811612100 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7efcc40229c0 con 0x7efcd0100e40 2023-12-14T18:25:34.485 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.424+0000 7efcccff9700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efcb80a11e0 0x7efcb80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:34.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.424+0000 7efccf7fe700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efcb80a11e0 0x7efcb80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:34.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.424+0000 7efcccff9700 1 -- 172.21.15.93:0/3811612100 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7efcc40ce670 con 0x7efcd0100e40 2023-12-14T18:25:34.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.424+0000 7efccf7fe700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efcb80a11e0 0x7efcb80a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efcc0006fd0 tx=0x7efcc0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:34.486 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.427+0000 7efcccff9700 1 -- 172.21.15.93:0/3811612100 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7efcc4097b00 con 0x7efcd0100e40 2023-12-14T18:25:34.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:34 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2469465404' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2023-12-14T18:25:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.621+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) v1 -- 0x7efcd0197860 con 0x7efcd0100e40 2023-12-14T18:25:34.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.621+0000 7efcccff9700 1 -- 172.21.15.93:0/3811612100 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) v1 ==== 75+0+13 (secure 0 0 0) 0x7efcc409b120 con 0x7efcd0100e40 2023-12-14T18:25:34.624 INFO:teuthology.orchestra.run.smithi093.stdout:287762808857 2023-12-14T18:25:34.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.623+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efcb80a11e0 msgr2=0x7efcb80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.623+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efcb80a11e0 0x7efcb80a3690 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7efcc0006fd0 tx=0x7efcc0008040 comp rx=0 tx=0).stop 2023-12-14T18:25:34.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.623+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100e40 msgr2=0x7efcd01969a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.623+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100e40 0x7efcd01969a0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efcc4006010 tx=0x7efcc400f7b0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.624+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 shutdown_connections 2023-12-14T18:25:34.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.624+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7efcb80a11e0 0x7efcb80a3690 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.624+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7efcd0106370 0x7efcd019ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.624+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7efcd0100490 0x7efcd0196460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.624+0000 7efcd5ee3700 1 --2- 172.21.15.93:0/3811612100 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7efcd0100e40 0x7efcd01969a0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.624+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 >> 172.21.15.93:0/3811612100 conn(0x7efcd0078430 msgr2=0x7efcd0103ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:34.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.624+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 shutdown_connections 2023-12-14T18:25:34.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.624+0000 7efcd5ee3700 1 -- 172.21.15.93:0/3811612100 wait complete. 2023-12-14T18:25:34.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3683797149 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf41005b0 msgr2=0x7f4cf4100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3683797149 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf41005b0 0x7f4cf4100980 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80099c0 tx=0x7f4ce8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3683797149 shutdown_connections 2023-12-14T18:25:34.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3683797149 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf41005b0 0x7f4cf4100980 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3683797149 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4105650 0x7f4cf410d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3683797149 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cf4100f50 0x7f4cf4104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3683797149 >> 172.21.15.93:0/3683797149 conn(0x7f4cf4077110 msgr2=0x7f4cf4077510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:34.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3683797149 shutdown_connections 2023-12-14T18:25:34.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3683797149 wait complete. 2023-12-14T18:25:34.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 Processor -- start 2023-12-14T18:25:34.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.661+0000 7f4cfb1b7700 1 -- start start 2023-12-14T18:25:34.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cfb1b7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cf41005b0 0x7f4cf41963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:34.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cfb1b7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100f50 0x7f4cf4196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:34.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cfb1b7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4105650 0x7f4cf419ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:34.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cfb1b7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf419b390 con 0x7f4cf4100f50 2023-12-14T18:25:34.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cfb1b7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf419b500 con 0x7f4cf4105650 2023-12-14T18:25:34.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cfb1b7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4cf419b6c0 con 0x7f4cf41005b0 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100f50 0x7f4cf4196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100f50 0x7f4cf4196910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47640/0 (socket says 172.21.15.93:47640) 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf3fff700 1 -- 172.21.15.93:0/3444619522 learned_addr learned my addr 172.21.15.93:0/3444619522 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf9754700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4105650 0x7f4cf419ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf8f53700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cf41005b0 0x7f4cf41963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf3fff700 1 -- 172.21.15.93:0/3444619522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cf41005b0 msgr2=0x7f4cf41963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf3fff700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cf41005b0 0x7f4cf41963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf3fff700 1 -- 172.21.15.93:0/3444619522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4105650 msgr2=0x7f4cf419ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf3fff700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4105650 0x7f4cf419ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf3fff700 1 -- 172.21.15.93:0/3444619522 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ce80095f0 con 0x7f4cf4100f50 2023-12-14T18:25:34.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf3fff700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100f50 0x7f4cf4196910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000e970 tx=0x7f4ce000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:34.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf1ffb700 1 -- 172.21.15.93:0/3444619522 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0009790 con 0x7f4cf4100f50 2023-12-14T18:25:34.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4cf41103e0 con 0x7f4cf4100f50 2023-12-14T18:25:34.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf1ffb700 1 -- 172.21.15.93:0/3444619522 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ce000eec0 con 0x7f4cf4100f50 2023-12-14T18:25:34.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.662+0000 7f4cf1ffb700 1 -- 172.21.15.93:0/3444619522 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ce0005590 con 0x7f4cf4100f50 2023-12-14T18:25:34.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.663+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4cf4110250 con 0x7f4cf4100f50 2023-12-14T18:25:34.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.664+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4cf4110570 con 0x7f4cf4100f50 2023-12-14T18:25:34.668 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.664+0000 7f4cf1ffb700 1 -- 172.21.15.93:0/3444619522 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4ce000cbd0 con 0x7f4cf4100f50 2023-12-14T18:25:34.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.665+0000 7f4cf1ffb700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ce40a11e0 0x7f4ce40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:34.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.665+0000 7f4cf8f53700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ce40a11e0 0x7f4ce40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:34.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.665+0000 7f4cf1ffb700 1 -- 172.21.15.93:0/3444619522 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4ce0014070 con 0x7f4cf4100f50 2023-12-14T18:25:34.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.666+0000 7f4cf8f53700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ce40a11e0 0x7f4ce40a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009990 tx=0x7f4ce801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:34.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.672+0000 7f4cf1ffb700 1 -- 172.21.15.93:0/3444619522 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4ce00962d0 con 0x7f4cf4100f50 2023-12-14T18:25:34.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.879+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f4cf4066ea0 con 0x7f4cf4100f50 2023-12-14T18:25:34.882 INFO:teuthology.orchestra.run.smithi093.stdout:184683593765 2023-12-14T18:25:34.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.880+0000 7f4cf1ffb700 1 -- 172.21.15.93:0/3444619522 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f4ce0004ef0 con 0x7f4cf4100f50 2023-12-14T18:25:34.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ce40a11e0 msgr2=0x7f4ce40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ce40a11e0 0x7f4ce40a3690 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8009990 tx=0x7f4ce801a040 comp rx=0 tx=0).stop 2023-12-14T18:25:34.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100f50 msgr2=0x7f4cf4196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:34.884 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100f50 0x7f4cf4196910 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4ce000e970 tx=0x7f4ce000ec80 comp rx=0 tx=0).stop 2023-12-14T18:25:34.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 shutdown_connections 2023-12-14T18:25:34.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ce40a11e0 0x7f4ce40a3690 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4cf4105650 0x7f4cf419ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4cf41005b0 0x7f4cf41963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 --2- 172.21.15.93:0/3444619522 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4cf4100f50 0x7f4cf4196910 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:34.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 >> 172.21.15.93:0/3444619522 conn(0x7f4cf4077110 msgr2=0x7f4cf40fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:34.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 shutdown_connections 2023-12-14T18:25:34.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:34.882+0000 7f4cfb1b7700 1 -- 172.21.15.93:0/3444619522 wait complete. 2023-12-14T18:25:35.044 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808855 got 287762808857 for osd.10 2023-12-14T18:25:35.044 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:35.172 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:35 smithi093 ceph-mon[42070]: pgmap v253: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:25:35.172 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:35 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3811612100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-14T18:25:35.172 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:35 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:35.172 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:35 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3444619522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-14T18:25:35.276 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593762 got 184683593765 for osd.6 2023-12-14T18:25:35.277 DEBUG:teuthology.parallel:result is None 2023-12-14T18:25:35.277 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2023-12-14T18:25:35.277 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph pg dump --format=json 2023-12-14T18:25:35.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:35 smithi097 ceph-mon[43486]: pgmap v253: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:25:35.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:35 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3811612100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-14T18:25:35.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:35 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:35.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:35 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3444619522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-14T18:25:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:35 smithi160 ceph-mon[43840]: pgmap v253: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:25:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:35 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3811612100' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2023-12-14T18:25:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:35 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:25:35.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:35 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3444619522' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2023-12-14T18:25:35.757 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:36.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.812+0000 7f4ed88a1700 1 -- 172.21.15.93:0/2248427340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 msgr2=0x7f4ed410a370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:36.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.812+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/2248427340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 0x7f4ed410a370 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009990 tx=0x7f4ec8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.812+0000 7f4ed88a1700 1 -- 172.21.15.93:0/2248427340 shutdown_connections 2023-12-14T18:25:36.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.812+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/2248427340 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed410a8b0 0x7f4ed410ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.812+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/2248427340 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed4101390 0x7f4ed4101760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.812+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/2248427340 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 0x7f4ed410a370 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.812+0000 7f4ed88a1700 1 -- 172.21.15.93:0/2248427340 >> 172.21.15.93:0/2248427340 conn(0x7f4ed40770a0 msgr2=0x7f4ed40774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:36.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.812+0000 7f4ed88a1700 1 -- 172.21.15.93:0/2248427340 shutdown_connections 2023-12-14T18:25:36.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.813+0000 7f4ed88a1700 1 -- 172.21.15.93:0/2248427340 wait complete. 2023-12-14T18:25:36.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.813+0000 7f4ed88a1700 1 Processor -- start 2023-12-14T18:25:36.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.813+0000 7f4ed88a1700 1 -- start start 2023-12-14T18:25:36.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed88a1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed4101390 0x7f4ed40ff1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:36.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed88a1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 0x7f4ed40ff700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:36.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed88a1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed410a8b0 0x7f4ed40ffc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:36.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed88a1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41a4910 con 0x7f4ed4101ca0 2023-12-14T18:25:36.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed88a1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41a4a80 con 0x7f4ed410a8b0 2023-12-14T18:25:36.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed88a1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed41a4c40 con 0x7f4ed4101390 2023-12-14T18:25:36.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 0x7f4ed40ff700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 0x7f4ed40ff700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47656/0 (socket says 172.21.15.93:47656) 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed1d9b700 1 -- 172.21.15.93:0/298907367 learned_addr learned my addr 172.21.15.93:0/298907367 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed259c700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed4101390 0x7f4ed40ff1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed2d9d700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed410a8b0 0x7f4ed40ffc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed1d9b700 1 -- 172.21.15.93:0/298907367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed4101390 msgr2=0x7f4ed40ff1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed1d9b700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed4101390 0x7f4ed40ff1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed1d9b700 1 -- 172.21.15.93:0/298907367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed410a8b0 msgr2=0x7f4ed40ffc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed1d9b700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed410a8b0 0x7f4ed40ffc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed1d9b700 1 -- 172.21.15.93:0/298907367 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec80095f0 con 0x7f4ed4101ca0 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed1d9b700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 0x7f4ed40ff700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8000c00 tx=0x7f4ec8009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ec37fe700 1 -- 172.21.15.93:0/298907367 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec801d070 con 0x7f4ed4101ca0 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed4110330 con 0x7f4ed4101ca0 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ec37fe700 1 -- 172.21.15.93:0/298907367 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4ec8022470 con 0x7f4ed4101ca0 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ec37fe700 1 -- 172.21.15.93:0/298907367 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8017650 con 0x7f4ed4101ca0 2023-12-14T18:25:36.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.814+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed41101a0 con 0x7f4ed4101ca0 2023-12-14T18:25:36.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.816+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed41104c0 con 0x7f4ed4101ca0 2023-12-14T18:25:36.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.817+0000 7f4ec37fe700 1 -- 172.21.15.93:0/298907367 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f4ec8022850 con 0x7f4ed4101ca0 2023-12-14T18:25:36.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.817+0000 7f4ec37fe700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ebc0a11e0 0x7f4ebc0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:36.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.817+0000 7f4ed259c700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ebc0a11e0 0x7f4ebc0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:36.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.817+0000 7f4ec37fe700 1 -- 172.21.15.93:0/298907367 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 8730+0+0 (secure 0 0 0) 0x7f4ec80cdfe0 con 0x7f4ed4101ca0 2023-12-14T18:25:36.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.818+0000 7f4ed259c700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ebc0a11e0 0x7f4ebc0a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009c60 tx=0x7f4ec4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:36.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:36.826+0000 7f4ec37fe700 1 -- 172.21.15.93:0/298907367 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4ec8097580 con 0x7f4ed4101ca0 2023-12-14T18:25:37.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.031+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f4ed40fc420 con 0x7f4ebc0a11e0 2023-12-14T18:25:37.045 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.043+0000 7f4ec37fe700 1 -- 172.21.15.93:0/298907367 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423223 (secure 0 0 0) 0x7f4ed40fc420 con 0x7f4ebc0a11e0 2023-12-14T18:25:37.050 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:37.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ebc0a11e0 msgr2=0x7f4ebc0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:37.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ebc0a11e0 0x7f4ebc0a3690 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4ec4009c60 tx=0x7f4ec4009260 comp rx=0 tx=0).stop 2023-12-14T18:25:37.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 msgr2=0x7f4ed40ff700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:37.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 0x7f4ed40ff700 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8000c00 tx=0x7f4ec8009580 comp rx=0 tx=0).stop 2023-12-14T18:25:37.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 shutdown_connections 2023-12-14T18:25:37.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f4ebc0a11e0 0x7f4ebc0a3690 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:37.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4ed410a8b0 0x7f4ed40ffc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:37.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4ed4101390 0x7f4ed40ff1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:37.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 --2- 172.21.15.93:0/298907367 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4ed4101ca0 0x7f4ed40ff700 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:37.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 >> 172.21.15.93:0/298907367 conn(0x7f4ed40770a0 msgr2=0x7f4ed40fbd10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:37.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 shutdown_connections 2023-12-14T18:25:37.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:37.047+0000 7f4ed88a1700 1 -- 172.21.15.93:0/298907367 wait complete. 2023-12-14T18:25:37.054 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-12-14T18:25:37.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: pgmap v254: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:36 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: pgmap v254: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:36 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]: dispatch 2023-12-14T18:25:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]: dispatch 2023-12-14T18:25:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]: dispatch 2023-12-14T18:25:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]: dispatch 2023-12-14T18:25:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]: dispatch 2023-12-14T18:25:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]: dispatch 2023-12-14T18:25:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2023-12-14T18:25:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]: dispatch 2023-12-14T18:25:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]: dispatch 2023-12-14T18:25:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]: dispatch 2023-12-14T18:25:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: pgmap v254: 129 pgs: 129 active+clean; 638 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:25:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:36 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:37.414 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":255,"stamp":"2023-12-14T18:25:36.837120+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655354,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":372,"num_read_kb":308,"num_write":336,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79676,"kb_used_data":11228,"kb_used_omap":0,"kb_used_meta":68352,"kb_avail":1124862148,"statfs":{"total":1151940427776,"available":1151858839552,"internally_reserved":0,"allocated":11497472,"data_stored":7444643,"data_compressed":246693,"data_compressed_allocated":331776,"data_compressed_original":2162688,"omap_allocated":0,"internal_metadata":69992448},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":71,"num_read_kb":41,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.879561"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939919+0000","last_change":"2023-12-14T18:24:36.587133+0000","last_active":"2023-12-14T18:24:41.939919+0000","last_peered":"2023-12-14T18:24:41.939919+0000","last_clean":"2023-12-14T18:24:41.939919+0000","last_became_active":"2023-12-14T18:24:36.586222+0000","last_became_peered":"2023-12-14T18:24:36.586222+0000","last_unstale":"2023-12-14T18:24:41.939919+0000","last_undegraded":"2023-12-14T18:24:41.939919+0000","last_fullsized":"2023-12-14T18:24:41.939919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:51:38.922525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939529+0000","last_change":"2023-12-14T18:24:36.589027+0000","last_active":"2023-12-14T18:24:41.939529+0000","last_peered":"2023-12-14T18:24:41.939529+0000","last_clean":"2023-12-14T18:24:41.939529+0000","last_became_active":"2023-12-14T18:24:36.588785+0000","last_became_peered":"2023-12-14T18:24:36.588785+0000","last_unstale":"2023-12-14T18:24:41.939529+0000","last_undegraded":"2023-12-14T18:24:41.939529+0000","last_fullsized":"2023-12-14T18:24:41.939529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:42.987957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939327+0000","last_change":"2023-12-14T18:24:36.584015+0000","last_active":"2023-12-14T18:24:41.939327+0000","last_peered":"2023-12-14T18:24:41.939327+0000","last_clean":"2023-12-14T18:24:41.939327+0000","last_became_active":"2023-12-14T18:24:36.583735+0000","last_became_peered":"2023-12-14T18:24:36.583735+0000","last_unstale":"2023-12-14T18:24:41.939327+0000","last_undegraded":"2023-12-14T18:24:41.939327+0000","last_fullsized":"2023-12-14T18:24:41.939327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:20:02.823540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940899+0000","last_change":"2023-12-14T18:24:36.584553+0000","last_active":"2023-12-14T18:24:41.940899+0000","last_peered":"2023-12-14T18:24:41.940899+0000","last_clean":"2023-12-14T18:24:41.940899+0000","last_became_active":"2023-12-14T18:24:36.583827+0000","last_became_peered":"2023-12-14T18:24:36.583827+0000","last_unstale":"2023-12-14T18:24:41.940899+0000","last_undegraded":"2023-12-14T18:24:41.940899+0000","last_fullsized":"2023-12-14T18:24:41.940899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:32.928164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095366+0000","last_change":"2023-12-14T18:24:36.578128+0000","last_active":"2023-12-14T18:24:41.095366+0000","last_peered":"2023-12-14T18:24:41.095366+0000","last_clean":"2023-12-14T18:24:41.095366+0000","last_became_active":"2023-12-14T18:24:36.577784+0000","last_became_peered":"2023-12-14T18:24:36.577784+0000","last_unstale":"2023-12-14T18:24:41.095366+0000","last_undegraded":"2023-12-14T18:24:41.095366+0000","last_fullsized":"2023-12-14T18:24:41.095366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:17.890524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940347+0000","last_change":"2023-12-14T18:24:36.588666+0000","last_active":"2023-12-14T18:24:41.940347+0000","last_peered":"2023-12-14T18:24:41.940347+0000","last_clean":"2023-12-14T18:24:41.940347+0000","last_became_active":"2023-12-14T18:24:36.588291+0000","last_became_peered":"2023-12-14T18:24:36.588291+0000","last_unstale":"2023-12-14T18:24:41.940347+0000","last_undegraded":"2023-12-14T18:24:41.940347+0000","last_fullsized":"2023-12-14T18:24:41.940347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:10:59.591257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941626+0000","last_change":"2023-12-14T18:24:36.586637+0000","last_active":"2023-12-14T18:24:41.941626+0000","last_peered":"2023-12-14T18:24:41.941626+0000","last_clean":"2023-12-14T18:24:41.941626+0000","last_became_active":"2023-12-14T18:24:36.583055+0000","last_became_peered":"2023-12-14T18:24:36.583055+0000","last_unstale":"2023-12-14T18:24:41.941626+0000","last_undegraded":"2023-12-14T18:24:41.941626+0000","last_fullsized":"2023-12-14T18:24:41.941626+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:31.010603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998916+0000","last_change":"2023-12-14T18:24:36.580206+0000","last_active":"2023-12-14T18:24:40.998916+0000","last_peered":"2023-12-14T18:24:40.998916+0000","last_clean":"2023-12-14T18:24:40.998916+0000","last_became_active":"2023-12-14T18:24:36.579904+0000","last_became_peered":"2023-12-14T18:24:36.579904+0000","last_unstale":"2023-12-14T18:24:40.998916+0000","last_undegraded":"2023-12-14T18:24:40.998916+0000","last_fullsized":"2023-12-14T18:24:40.998916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:15.182919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000918+0000","last_change":"2023-12-14T18:24:36.589890+0000","last_active":"2023-12-14T18:24:41.000918+0000","last_peered":"2023-12-14T18:24:41.000918+0000","last_clean":"2023-12-14T18:24:41.000918+0000","last_became_active":"2023-12-14T18:24:36.588999+0000","last_became_peered":"2023-12-14T18:24:36.588999+0000","last_unstale":"2023-12-14T18:24:41.000918+0000","last_undegraded":"2023-12-14T18:24:41.000918+0000","last_fullsized":"2023-12-14T18:24:41.000918+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:59:07.801717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941211+0000","last_change":"2023-12-14T18:24:36.589565+0000","last_active":"2023-12-14T18:24:41.941211+0000","last_peered":"2023-12-14T18:24:41.941211+0000","last_clean":"2023-12-14T18:24:41.941211+0000","last_became_active":"2023-12-14T18:24:36.588822+0000","last_became_peered":"2023-12-14T18:24:36.588822+0000","last_unstale":"2023-12-14T18:24:41.941211+0000","last_undegraded":"2023-12-14T18:24:41.941211+0000","last_fullsized":"2023-12-14T18:24:41.941211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:20:30.545830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941183+0000","last_change":"2023-12-14T18:24:36.593208+0000","last_active":"2023-12-14T18:24:41.941183+0000","last_peered":"2023-12-14T18:24:41.941183+0000","last_clean":"2023-12-14T18:24:41.941183+0000","last_became_active":"2023-12-14T18:24:36.586394+0000","last_became_peered":"2023-12-14T18:24:36.586394+0000","last_unstale":"2023-12-14T18:24:41.941183+0000","last_undegraded":"2023-12-14T18:24:41.941183+0000","last_fullsized":"2023-12-14T18:24:41.941183+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:19:50.115165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940763+0000","last_change":"2023-12-14T18:24:36.594254+0000","last_active":"2023-12-14T18:24:41.940763+0000","last_peered":"2023-12-14T18:24:41.940763+0000","last_clean":"2023-12-14T18:24:41.940763+0000","last_became_active":"2023-12-14T18:24:36.593889+0000","last_became_peered":"2023-12-14T18:24:36.593889+0000","last_unstale":"2023-12-14T18:24:41.940763+0000","last_undegraded":"2023-12-14T18:24:41.940763+0000","last_fullsized":"2023-12-14T18:24:41.940763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:50.487280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939911+0000","last_change":"2023-12-14T18:24:36.584971+0000","last_active":"2023-12-14T18:24:41.939911+0000","last_peered":"2023-12-14T18:24:41.939911+0000","last_clean":"2023-12-14T18:24:41.939911+0000","last_became_active":"2023-12-14T18:24:36.584695+0000","last_became_peered":"2023-12-14T18:24:36.584695+0000","last_unstale":"2023-12-14T18:24:41.939911+0000","last_undegraded":"2023-12-14T18:24:41.939911+0000","last_fullsized":"2023-12-14T18:24:41.939911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:43:12.153321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939181+0000","last_change":"2023-12-14T18:24:36.589692+0000","last_active":"2023-12-14T18:24:41.939181+0000","last_peered":"2023-12-14T18:24:41.939181+0000","last_clean":"2023-12-14T18:24:41.939181+0000","last_became_active":"2023-12-14T18:24:36.589415+0000","last_became_peered":"2023-12-14T18:24:36.589415+0000","last_unstale":"2023-12-14T18:24:41.939181+0000","last_undegraded":"2023-12-14T18:24:41.939181+0000","last_fullsized":"2023-12-14T18:24:41.939181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:17:55.715589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.938924+0000","last_change":"2023-12-14T18:24:36.586911+0000","last_active":"2023-12-14T18:24:41.938924+0000","last_peered":"2023-12-14T18:24:41.938924+0000","last_clean":"2023-12-14T18:24:41.938924+0000","last_became_active":"2023-12-14T18:24:36.586600+0000","last_became_peered":"2023-12-14T18:24:36.586600+0000","last_unstale":"2023-12-14T18:24:41.938924+0000","last_undegraded":"2023-12-14T18:24:41.938924+0000","last_fullsized":"2023-12-14T18:24:41.938924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:00:56.150982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000169+0000","last_change":"2023-12-14T18:24:36.584879+0000","last_active":"2023-12-14T18:24:41.000169+0000","last_peered":"2023-12-14T18:24:41.000169+0000","last_clean":"2023-12-14T18:24:41.000169+0000","last_became_active":"2023-12-14T18:24:36.584566+0000","last_became_peered":"2023-12-14T18:24:36.584566+0000","last_unstale":"2023-12-14T18:24:41.000169+0000","last_undegraded":"2023-12-14T18:24:41.000169+0000","last_fullsized":"2023-12-14T18:24:41.000169+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:23:00.946127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939646+0000","last_change":"2023-12-14T18:24:36.587420+0000","last_active":"2023-12-14T18:24:41.939646+0000","last_peered":"2023-12-14T18:24:41.939646+0000","last_clean":"2023-12-14T18:24:41.939646+0000","last_became_active":"2023-12-14T18:24:36.586732+0000","last_became_peered":"2023-12-14T18:24:36.586732+0000","last_unstale":"2023-12-14T18:24:41.939646+0000","last_undegraded":"2023-12-14T18:24:41.939646+0000","last_fullsized":"2023-12-14T18:24:41.939646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:06:51.142371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093222+0000","last_change":"2023-12-14T18:24:36.583412+0000","last_active":"2023-12-14T18:24:41.093222+0000","last_peered":"2023-12-14T18:24:41.093222+0000","last_clean":"2023-12-14T18:24:41.093222+0000","last_became_active":"2023-12-14T18:24:36.583022+0000","last_became_peered":"2023-12-14T18:24:36.583022+0000","last_unstale":"2023-12-14T18:24:41.093222+0000","last_undegraded":"2023-12-14T18:24:41.093222+0000","last_fullsized":"2023-12-14T18:24:41.093222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:35:26.281488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001176+0000","last_change":"2023-12-14T18:24:36.575232+0000","last_active":"2023-12-14T18:24:41.001176+0000","last_peered":"2023-12-14T18:24:41.001176+0000","last_clean":"2023-12-14T18:24:41.001176+0000","last_became_active":"2023-12-14T18:24:36.574573+0000","last_became_peered":"2023-12-14T18:24:36.574573+0000","last_unstale":"2023-12-14T18:24:41.001176+0000","last_undegraded":"2023-12-14T18:24:41.001176+0000","last_fullsized":"2023-12-14T18:24:41.001176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:09:44.789329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000573+0000","last_change":"2023-12-14T18:24:40.605077+0000","last_active":"2023-12-14T18:24:41.000573+0000","last_peered":"2023-12-14T18:24:41.000573+0000","last_clean":"2023-12-14T18:24:41.000573+0000","last_became_active":"2023-12-14T18:24:40.604646+0000","last_became_peered":"2023-12-14T18:24:40.604646+0000","last_unstale":"2023-12-14T18:24:41.000573+0000","last_undegraded":"2023-12-14T18:24:41.000573+0000","last_fullsized":"2023-12-14T18:24:41.000573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:46:49.342657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.768860+0000","last_change":"2023-12-14T18:24:36.177449+0000","last_active":"2023-12-14T18:25:00.768860+0000","last_peered":"2023-12-14T18:25:00.768860+0000","last_clean":"2023-12-14T18:25:00.768860+0000","last_became_active":"2023-12-14T18:24:36.177114+0000","last_became_peered":"2023-12-14T18:24:36.177114+0000","last_unstale":"2023-12-14T18:25:00.768860+0000","last_undegraded":"2023-12-14T18:25:00.768860+0000","last_fullsized":"2023-12-14T18:25:00.768860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:30.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999464+0000","last_change":"2023-12-14T18:24:36.574204+0000","last_active":"2023-12-14T18:24:40.999464+0000","last_peered":"2023-12-14T18:24:40.999464+0000","last_clean":"2023-12-14T18:24:40.999464+0000","last_became_active":"2023-12-14T18:24:36.573923+0000","last_became_peered":"2023-12-14T18:24:36.573923+0000","last_unstale":"2023-12-14T18:24:40.999464+0000","last_undegraded":"2023-12-14T18:24:40.999464+0000","last_fullsized":"2023-12-14T18:24:40.999464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:20:32.637706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939617+0000","last_change":"2023-12-14T18:24:40.600254+0000","last_active":"2023-12-14T18:24:41.939617+0000","last_peered":"2023-12-14T18:24:41.939617+0000","last_clean":"2023-12-14T18:24:41.939617+0000","last_became_active":"2023-12-14T18:24:40.599894+0000","last_became_peered":"2023-12-14T18:24:40.599894+0000","last_unstale":"2023-12-14T18:24:41.939617+0000","last_undegraded":"2023-12-14T18:24:41.939617+0000","last_fullsized":"2023-12-14T18:24:41.939617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:00.116137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999521+0000","last_change":"2023-12-14T18:24:36.170109+0000","last_active":"2023-12-14T18:24:40.999521+0000","last_peered":"2023-12-14T18:24:40.999521+0000","last_clean":"2023-12-14T18:24:40.999521+0000","last_became_active":"2023-12-14T18:24:36.169629+0000","last_became_peered":"2023-12-14T18:24:36.169629+0000","last_unstale":"2023-12-14T18:24:40.999521+0000","last_undegraded":"2023-12-14T18:24:40.999521+0000","last_fullsized":"2023-12-14T18:24:40.999521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:41:16.457493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939537+0000","last_change":"2023-12-14T18:24:36.582757+0000","last_active":"2023-12-14T18:24:41.939537+0000","last_peered":"2023-12-14T18:24:41.939537+0000","last_clean":"2023-12-14T18:24:41.939537+0000","last_became_active":"2023-12-14T18:24:36.582457+0000","last_became_peered":"2023-12-14T18:24:36.582457+0000","last_unstale":"2023-12-14T18:24:41.939537+0000","last_undegraded":"2023-12-14T18:24:41.939537+0000","last_fullsized":"2023-12-14T18:24:41.939537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:29:05.124428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940384+0000","last_change":"2023-12-14T18:24:40.599695+0000","last_active":"2023-12-14T18:24:41.940384+0000","last_peered":"2023-12-14T18:24:41.940384+0000","last_clean":"2023-12-14T18:24:41.940384+0000","last_became_active":"2023-12-14T18:24:40.599239+0000","last_became_peered":"2023-12-14T18:24:40.599239+0000","last_unstale":"2023-12-14T18:24:41.940384+0000","last_undegraded":"2023-12-14T18:24:41.940384+0000","last_fullsized":"2023-12-14T18:24:41.940384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:40.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.766757+0000","last_change":"2023-12-14T18:24:36.177946+0000","last_active":"2023-12-14T18:25:00.766757+0000","last_peered":"2023-12-14T18:25:00.766757+0000","last_clean":"2023-12-14T18:25:00.766757+0000","last_became_active":"2023-12-14T18:24:36.177480+0000","last_became_peered":"2023-12-14T18:24:36.177480+0000","last_unstale":"2023-12-14T18:25:00.766757+0000","last_undegraded":"2023-12-14T18:25:00.766757+0000","last_fullsized":"2023-12-14T18:25:00.766757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:42:02.447975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000436+0000","last_change":"2023-12-14T18:24:36.587951+0000","last_active":"2023-12-14T18:24:41.000436+0000","last_peered":"2023-12-14T18:24:41.000436+0000","last_clean":"2023-12-14T18:24:41.000436+0000","last_became_active":"2023-12-14T18:24:36.587539+0000","last_became_peered":"2023-12-14T18:24:36.587539+0000","last_unstale":"2023-12-14T18:24:41.000436+0000","last_undegraded":"2023-12-14T18:24:41.000436+0000","last_fullsized":"2023-12-14T18:24:41.000436+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:53:03.285569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941747+0000","last_change":"2023-12-14T18:24:40.600008+0000","last_active":"2023-12-14T18:24:41.941747+0000","last_peered":"2023-12-14T18:24:41.941747+0000","last_clean":"2023-12-14T18:24:41.941747+0000","last_became_active":"2023-12-14T18:24:40.599596+0000","last_became_peered":"2023-12-14T18:24:40.599596+0000","last_unstale":"2023-12-14T18:24:41.941747+0000","last_undegraded":"2023-12-14T18:24:41.941747+0000","last_fullsized":"2023-12-14T18:24:41.941747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:10:10.770024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095629+0000","last_change":"2023-12-14T18:24:36.176569+0000","last_active":"2023-12-14T18:24:41.095629+0000","last_peered":"2023-12-14T18:24:41.095629+0000","last_clean":"2023-12-14T18:24:41.095629+0000","last_became_active":"2023-12-14T18:24:36.176043+0000","last_became_peered":"2023-12-14T18:24:36.176043+0000","last_unstale":"2023-12-14T18:24:41.095629+0000","last_undegraded":"2023-12-14T18:24:41.095629+0000","last_fullsized":"2023-12-14T18:24:41.095629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:10:44.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149101+0000","last_change":"2023-12-14T18:24:36.593761+0000","last_active":"2023-12-14T18:24:41.149101+0000","last_peered":"2023-12-14T18:24:41.149101+0000","last_clean":"2023-12-14T18:24:41.149101+0000","last_became_active":"2023-12-14T18:24:36.593452+0000","last_became_peered":"2023-12-14T18:24:36.593452+0000","last_unstale":"2023-12-14T18:24:41.149101+0000","last_undegraded":"2023-12-14T18:24:41.149101+0000","last_fullsized":"2023-12-14T18:24:41.149101+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:56:34.794908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001069+0000","last_change":"2023-12-14T18:24:40.598064+0000","last_active":"2023-12-14T18:24:41.001069+0000","last_peered":"2023-12-14T18:24:41.001069+0000","last_clean":"2023-12-14T18:24:41.001069+0000","last_became_active":"2023-12-14T18:24:40.597672+0000","last_became_peered":"2023-12-14T18:24:40.597672+0000","last_unstale":"2023-12-14T18:24:41.001069+0000","last_undegraded":"2023-12-14T18:24:41.001069+0000","last_fullsized":"2023-12-14T18:24:41.001069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:41:54.138227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":63,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:30.805351+0000","last_change":"2023-12-14T18:24:36.176288+0000","last_active":"2023-12-14T18:25:30.805351+0000","last_peered":"2023-12-14T18:25:30.805351+0000","last_clean":"2023-12-14T18:25:30.805351+0000","last_became_active":"2023-12-14T18:24:36.175965+0000","last_became_peered":"2023-12-14T18:24:36.175965+0000","last_unstale":"2023-12-14T18:25:30.805351+0000","last_undegraded":"2023-12-14T18:25:30.805351+0000","last_fullsized":"2023-12-14T18:25:30.805351+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:00:50.888829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095492+0000","last_change":"2023-12-14T18:24:36.583687+0000","last_active":"2023-12-14T18:24:41.095492+0000","last_peered":"2023-12-14T18:24:41.095492+0000","last_clean":"2023-12-14T18:24:41.095492+0000","last_became_active":"2023-12-14T18:24:36.583047+0000","last_became_peered":"2023-12-14T18:24:36.583047+0000","last_unstale":"2023-12-14T18:24:41.095492+0000","last_undegraded":"2023-12-14T18:24:41.095492+0000","last_fullsized":"2023-12-14T18:24:41.095492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:26:10.095059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001102+0000","last_change":"2023-12-14T18:24:40.605217+0000","last_active":"2023-12-14T18:24:41.001102+0000","last_peered":"2023-12-14T18:24:41.001102+0000","last_clean":"2023-12-14T18:24:41.001102+0000","last_became_active":"2023-12-14T18:24:40.604807+0000","last_became_peered":"2023-12-14T18:24:40.604807+0000","last_unstale":"2023-12-14T18:24:41.001102+0000","last_undegraded":"2023-12-14T18:24:41.001102+0000","last_fullsized":"2023-12-14T18:24:41.001102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:48:27.699822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767627+0000","last_change":"2023-12-14T18:24:36.174875+0000","last_active":"2023-12-14T18:25:00.767627+0000","last_peered":"2023-12-14T18:25:00.767627+0000","last_clean":"2023-12-14T18:25:00.767627+0000","last_became_active":"2023-12-14T18:24:36.173984+0000","last_became_peered":"2023-12-14T18:24:36.173984+0000","last_unstale":"2023-12-14T18:25:00.767627+0000","last_undegraded":"2023-12-14T18:25:00.767627+0000","last_fullsized":"2023-12-14T18:25:00.767627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:24:27.753878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149194+0000","last_change":"2023-12-14T18:24:36.580563+0000","last_active":"2023-12-14T18:24:41.149194+0000","last_peered":"2023-12-14T18:24:41.149194+0000","last_clean":"2023-12-14T18:24:41.149194+0000","last_became_active":"2023-12-14T18:24:36.580196+0000","last_became_peered":"2023-12-14T18:24:36.580196+0000","last_unstale":"2023-12-14T18:24:41.149194+0000","last_undegraded":"2023-12-14T18:24:41.149194+0000","last_fullsized":"2023-12-14T18:24:41.149194+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:44:38.053372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939735+0000","last_change":"2023-12-14T18:24:40.601223+0000","last_active":"2023-12-14T18:24:41.939735+0000","last_peered":"2023-12-14T18:24:41.939735+0000","last_clean":"2023-12-14T18:24:41.939735+0000","last_became_active":"2023-12-14T18:24:40.600795+0000","last_became_peered":"2023-12-14T18:24:40.600795+0000","last_unstale":"2023-12-14T18:24:41.939735+0000","last_undegraded":"2023-12-14T18:24:41.939735+0000","last_fullsized":"2023-12-14T18:24:41.939735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:27:49.251012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767406+0000","last_change":"2023-12-14T18:24:36.173600+0000","last_active":"2023-12-14T18:25:00.767406+0000","last_peered":"2023-12-14T18:25:00.767406+0000","last_clean":"2023-12-14T18:25:00.767406+0000","last_became_active":"2023-12-14T18:24:36.173272+0000","last_became_peered":"2023-12-14T18:24:36.173272+0000","last_unstale":"2023-12-14T18:25:00.767406+0000","last_undegraded":"2023-12-14T18:25:00.767406+0000","last_fullsized":"2023-12-14T18:25:00.767406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:23.589714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998814+0000","last_change":"2023-12-14T18:24:36.575060+0000","last_active":"2023-12-14T18:24:40.998814+0000","last_peered":"2023-12-14T18:24:40.998814+0000","last_clean":"2023-12-14T18:24:40.998814+0000","last_became_active":"2023-12-14T18:24:36.574749+0000","last_became_peered":"2023-12-14T18:24:36.574749+0000","last_unstale":"2023-12-14T18:24:40.998814+0000","last_undegraded":"2023-12-14T18:24:40.998814+0000","last_fullsized":"2023-12-14T18:24:40.998814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:44:44.163762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939489+0000","last_change":"2023-12-14T18:24:40.605431+0000","last_active":"2023-12-14T18:24:41.939489+0000","last_peered":"2023-12-14T18:24:41.939489+0000","last_clean":"2023-12-14T18:24:41.939489+0000","last_became_active":"2023-12-14T18:24:40.604951+0000","last_became_peered":"2023-12-14T18:24:40.604951+0000","last_unstale":"2023-12-14T18:24:41.939489+0000","last_undegraded":"2023-12-14T18:24:41.939489+0000","last_fullsized":"2023-12-14T18:24:41.939489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:46:36.552203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148653+0000","last_change":"2023-12-14T18:24:36.175493+0000","last_active":"2023-12-14T18:24:41.148653+0000","last_peered":"2023-12-14T18:24:41.148653+0000","last_clean":"2023-12-14T18:24:41.148653+0000","last_became_active":"2023-12-14T18:24:36.175205+0000","last_became_peered":"2023-12-14T18:24:36.175205+0000","last_unstale":"2023-12-14T18:24:41.148653+0000","last_undegraded":"2023-12-14T18:24:41.148653+0000","last_fullsized":"2023-12-14T18:24:41.148653+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:52:40.611249+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092822+0000","last_change":"2023-12-14T18:24:36.587218+0000","last_active":"2023-12-14T18:24:41.092822+0000","last_peered":"2023-12-14T18:24:41.092822+0000","last_clean":"2023-12-14T18:24:41.092822+0000","last_became_active":"2023-12-14T18:24:36.586910+0000","last_became_peered":"2023-12-14T18:24:36.586910+0000","last_unstale":"2023-12-14T18:24:41.092822+0000","last_undegraded":"2023-12-14T18:24:41.092822+0000","last_fullsized":"2023-12-14T18:24:41.092822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:04.691462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999151+0000","last_change":"2023-12-14T18:24:40.598409+0000","last_active":"2023-12-14T18:24:40.999151+0000","last_peered":"2023-12-14T18:24:40.999151+0000","last_clean":"2023-12-14T18:24:40.999151+0000","last_became_active":"2023-12-14T18:24:40.598011+0000","last_became_peered":"2023-12-14T18:24:40.598011+0000","last_unstale":"2023-12-14T18:24:40.999151+0000","last_undegraded":"2023-12-14T18:24:40.999151+0000","last_fullsized":"2023-12-14T18:24:40.999151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:30.946044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:34.810413+0000","last_change":"2023-12-14T18:24:36.172277+0000","last_active":"2023-12-14T18:25:34.810413+0000","last_peered":"2023-12-14T18:25:34.810413+0000","last_clean":"2023-12-14T18:25:34.810413+0000","last_became_active":"2023-12-14T18:24:36.171907+0000","last_became_peered":"2023-12-14T18:24:36.171907+0000","last_unstale":"2023-12-14T18:25:34.810413+0000","last_undegraded":"2023-12-14T18:25:34.810413+0000","last_fullsized":"2023-12-14T18:25:34.810413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:45:12.002124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000922+0000","last_change":"2023-12-14T18:24:36.584783+0000","last_active":"2023-12-14T18:24:41.000922+0000","last_peered":"2023-12-14T18:24:41.000922+0000","last_clean":"2023-12-14T18:24:41.000922+0000","last_became_active":"2023-12-14T18:24:36.584424+0000","last_became_peered":"2023-12-14T18:24:36.584424+0000","last_unstale":"2023-12-14T18:24:41.000922+0000","last_undegraded":"2023-12-14T18:24:41.000922+0000","last_fullsized":"2023-12-14T18:24:41.000922+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:13:46.074921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092851+0000","last_change":"2023-12-14T18:24:40.603176+0000","last_active":"2023-12-14T18:24:41.092851+0000","last_peered":"2023-12-14T18:24:41.092851+0000","last_clean":"2023-12-14T18:24:41.092851+0000","last_became_active":"2023-12-14T18:24:40.602868+0000","last_became_peered":"2023-12-14T18:24:40.602868+0000","last_unstale":"2023-12-14T18:24:41.092851+0000","last_undegraded":"2023-12-14T18:24:41.092851+0000","last_fullsized":"2023-12-14T18:24:41.092851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:02:23.455860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.037799+0000","last_change":"2023-12-14T18:24:36.177638+0000","last_active":"2023-12-14T18:25:00.037799+0000","last_peered":"2023-12-14T18:25:00.037799+0000","last_clean":"2023-12-14T18:25:00.037799+0000","last_became_active":"2023-12-14T18:24:36.177335+0000","last_became_peered":"2023-12-14T18:24:36.177335+0000","last_unstale":"2023-12-14T18:25:00.037799+0000","last_undegraded":"2023-12-14T18:25:00.037799+0000","last_fullsized":"2023-12-14T18:25:00.037799+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:50:02.813317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939828+0000","last_change":"2023-12-14T18:24:36.577508+0000","last_active":"2023-12-14T18:24:41.939828+0000","last_peered":"2023-12-14T18:24:41.939828+0000","last_clean":"2023-12-14T18:24:41.939828+0000","last_became_active":"2023-12-14T18:24:36.577196+0000","last_became_peered":"2023-12-14T18:24:36.577196+0000","last_unstale":"2023-12-14T18:24:41.939828+0000","last_undegraded":"2023-12-14T18:24:41.939828+0000","last_fullsized":"2023-12-14T18:24:41.939828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:32:16.278415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941240+0000","last_change":"2023-12-14T18:24:40.594816+0000","last_active":"2023-12-14T18:24:41.941240+0000","last_peered":"2023-12-14T18:24:41.941240+0000","last_clean":"2023-12-14T18:24:41.941240+0000","last_became_active":"2023-12-14T18:24:40.594371+0000","last_became_peered":"2023-12-14T18:24:40.594371+0000","last_unstale":"2023-12-14T18:24:41.941240+0000","last_undegraded":"2023-12-14T18:24:41.941240+0000","last_fullsized":"2023-12-14T18:24:41.941240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:40:46.343094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039258+0000","last_change":"2023-12-14T18:24:36.174238+0000","last_active":"2023-12-14T18:25:00.039258+0000","last_peered":"2023-12-14T18:25:00.039258+0000","last_clean":"2023-12-14T18:25:00.039258+0000","last_became_active":"2023-12-14T18:24:36.173373+0000","last_became_peered":"2023-12-14T18:24:36.173373+0000","last_unstale":"2023-12-14T18:25:00.039258+0000","last_undegraded":"2023-12-14T18:25:00.039258+0000","last_fullsized":"2023-12-14T18:25:00.039258+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:40:50.640090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000757+0000","last_change":"2023-12-14T18:24:36.588589+0000","last_active":"2023-12-14T18:24:41.000757+0000","last_peered":"2023-12-14T18:24:41.000757+0000","last_clean":"2023-12-14T18:24:41.000757+0000","last_became_active":"2023-12-14T18:24:36.588269+0000","last_became_peered":"2023-12-14T18:24:36.588269+0000","last_unstale":"2023-12-14T18:24:41.000757+0000","last_undegraded":"2023-12-14T18:24:41.000757+0000","last_fullsized":"2023-12-14T18:24:41.000757+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:12.316505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092841+0000","last_change":"2023-12-14T18:24:40.601333+0000","last_active":"2023-12-14T18:24:41.092841+0000","last_peered":"2023-12-14T18:24:41.092841+0000","last_clean":"2023-12-14T18:24:41.092841+0000","last_became_active":"2023-12-14T18:24:40.600966+0000","last_became_peered":"2023-12-14T18:24:40.600966+0000","last_unstale":"2023-12-14T18:24:41.092841+0000","last_undegraded":"2023-12-14T18:24:41.092841+0000","last_fullsized":"2023-12-14T18:24:41.092841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:36.433276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":133,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:35.811430+0000","last_change":"2023-12-14T18:24:36.177299+0000","last_active":"2023-12-14T18:25:35.811430+0000","last_peered":"2023-12-14T18:25:35.811430+0000","last_clean":"2023-12-14T18:25:35.811430+0000","last_became_active":"2023-12-14T18:24:36.176971+0000","last_became_peered":"2023-12-14T18:24:36.176971+0000","last_unstale":"2023-12-14T18:25:35.811430+0000","last_undegraded":"2023-12-14T18:25:35.811430+0000","last_fullsized":"2023-12-14T18:25:35.811430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:02:46.083153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60836,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":80,"num_read_kb":99,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095663+0000","last_change":"2023-12-14T18:24:36.586315+0000","last_active":"2023-12-14T18:24:41.095663+0000","last_peered":"2023-12-14T18:24:41.095663+0000","last_clean":"2023-12-14T18:24:41.095663+0000","last_became_active":"2023-12-14T18:24:36.586000+0000","last_became_peered":"2023-12-14T18:24:36.586000+0000","last_unstale":"2023-12-14T18:24:41.095663+0000","last_undegraded":"2023-12-14T18:24:41.095663+0000","last_fullsized":"2023-12-14T18:24:41.095663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:19:55.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149539+0000","last_change":"2023-12-14T18:24:40.596973+0000","last_active":"2023-12-14T18:24:41.149539+0000","last_peered":"2023-12-14T18:24:41.149539+0000","last_clean":"2023-12-14T18:24:41.149539+0000","last_became_active":"2023-12-14T18:24:40.596355+0000","last_became_peered":"2023-12-14T18:24:40.596355+0000","last_unstale":"2023-12-14T18:24:41.149539+0000","last_undegraded":"2023-12-14T18:24:41.149539+0000","last_fullsized":"2023-12-14T18:24:41.149539+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:19:02.715446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767785+0000","last_change":"2023-12-14T18:24:36.175287+0000","last_active":"2023-12-14T18:25:00.767785+0000","last_peered":"2023-12-14T18:25:00.767785+0000","last_clean":"2023-12-14T18:25:00.767785+0000","last_became_active":"2023-12-14T18:24:36.174952+0000","last_became_peered":"2023-12-14T18:24:36.174952+0000","last_unstale":"2023-12-14T18:25:00.767785+0000","last_undegraded":"2023-12-14T18:25:00.767785+0000","last_fullsized":"2023-12-14T18:25:00.767785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:44:43.661729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940588+0000","last_change":"2023-12-14T18:24:36.582314+0000","last_active":"2023-12-14T18:24:41.940588+0000","last_peered":"2023-12-14T18:24:41.940588+0000","last_clean":"2023-12-14T18:24:41.940588+0000","last_became_active":"2023-12-14T18:24:36.581963+0000","last_became_peered":"2023-12-14T18:24:36.581963+0000","last_unstale":"2023-12-14T18:24:41.940588+0000","last_undegraded":"2023-12-14T18:24:41.940588+0000","last_fullsized":"2023-12-14T18:24:41.940588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:02:38.477328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939741+0000","last_change":"2023-12-14T18:24:40.599979+0000","last_active":"2023-12-14T18:24:41.939741+0000","last_peered":"2023-12-14T18:24:41.939741+0000","last_clean":"2023-12-14T18:24:41.939741+0000","last_became_active":"2023-12-14T18:24:40.599258+0000","last_became_peered":"2023-12-14T18:24:40.599258+0000","last_unstale":"2023-12-14T18:24:41.939741+0000","last_undegraded":"2023-12-14T18:24:41.939741+0000","last_fullsized":"2023-12-14T18:24:41.939741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:06.783797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039434+0000","last_change":"2023-12-14T18:24:36.175768+0000","last_active":"2023-12-14T18:25:00.039434+0000","last_peered":"2023-12-14T18:25:00.039434+0000","last_clean":"2023-12-14T18:25:00.039434+0000","last_became_active":"2023-12-14T18:24:36.174973+0000","last_became_peered":"2023-12-14T18:24:36.174973+0000","last_unstale":"2023-12-14T18:25:00.039434+0000","last_undegraded":"2023-12-14T18:25:00.039434+0000","last_fullsized":"2023-12-14T18:25:00.039434+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:54.191789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940662+0000","last_change":"2023-12-14T18:24:36.591856+0000","last_active":"2023-12-14T18:24:41.940662+0000","last_peered":"2023-12-14T18:24:41.940662+0000","last_clean":"2023-12-14T18:24:41.940662+0000","last_became_active":"2023-12-14T18:24:36.591554+0000","last_became_peered":"2023-12-14T18:24:36.591554+0000","last_unstale":"2023-12-14T18:24:41.940662+0000","last_undegraded":"2023-12-14T18:24:41.940662+0000","last_fullsized":"2023-12-14T18:24:41.940662+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:31:39.565049+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093250+0000","last_change":"2023-12-14T18:24:40.603794+0000","last_active":"2023-12-14T18:24:41.093250+0000","last_peered":"2023-12-14T18:24:41.093250+0000","last_clean":"2023-12-14T18:24:41.093250+0000","last_became_active":"2023-12-14T18:24:40.603311+0000","last_became_peered":"2023-12-14T18:24:40.603311+0000","last_unstale":"2023-12-14T18:24:41.093250+0000","last_undegraded":"2023-12-14T18:24:41.093250+0000","last_fullsized":"2023-12-14T18:24:41.093250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:11:14.354480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:33.832001+0000","last_change":"2023-12-14T18:24:36.177411+0000","last_active":"2023-12-14T18:25:33.832001+0000","last_peered":"2023-12-14T18:25:33.832001+0000","last_clean":"2023-12-14T18:25:33.832001+0000","last_became_active":"2023-12-14T18:24:36.177047+0000","last_became_peered":"2023-12-14T18:24:36.177047+0000","last_unstale":"2023-12-14T18:25:33.832001+0000","last_undegraded":"2023-12-14T18:25:33.832001+0000","last_fullsized":"2023-12-14T18:25:33.832001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:58:22.871429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939900+0000","last_change":"2023-12-14T18:24:36.597335+0000","last_active":"2023-12-14T18:24:41.939900+0000","last_peered":"2023-12-14T18:24:41.939900+0000","last_clean":"2023-12-14T18:24:41.939900+0000","last_became_active":"2023-12-14T18:24:36.596900+0000","last_became_peered":"2023-12-14T18:24:36.596900+0000","last_unstale":"2023-12-14T18:24:41.939900+0000","last_undegraded":"2023-12-14T18:24:41.939900+0000","last_fullsized":"2023-12-14T18:24:41.939900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:07:51.506541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940190+0000","last_change":"2023-12-14T18:24:40.607042+0000","last_active":"2023-12-14T18:24:41.940190+0000","last_peered":"2023-12-14T18:24:41.940190+0000","last_clean":"2023-12-14T18:24:41.940190+0000","last_became_active":"2023-12-14T18:24:40.606563+0000","last_became_peered":"2023-12-14T18:24:40.606563+0000","last_unstale":"2023-12-14T18:24:41.940190+0000","last_undegraded":"2023-12-14T18:24:41.940190+0000","last_fullsized":"2023-12-14T18:24:41.940190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:31:29.839031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.769004+0000","last_change":"2023-12-14T18:24:36.171994+0000","last_active":"2023-12-14T18:25:00.769004+0000","last_peered":"2023-12-14T18:25:00.769004+0000","last_clean":"2023-12-14T18:25:00.769004+0000","last_became_active":"2023-12-14T18:24:36.171661+0000","last_became_peered":"2023-12-14T18:24:36.171661+0000","last_unstale":"2023-12-14T18:25:00.769004+0000","last_undegraded":"2023-12-14T18:25:00.769004+0000","last_fullsized":"2023-12-14T18:25:00.769004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:33:33.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148674+0000","last_change":"2023-12-14T18:24:36.577818+0000","last_active":"2023-12-14T18:24:41.148674+0000","last_peered":"2023-12-14T18:24:41.148674+0000","last_clean":"2023-12-14T18:24:41.148674+0000","last_became_active":"2023-12-14T18:24:36.577356+0000","last_became_peered":"2023-12-14T18:24:36.577356+0000","last_unstale":"2023-12-14T18:24:41.148674+0000","last_undegraded":"2023-12-14T18:24:41.148674+0000","last_fullsized":"2023-12-14T18:24:41.148674+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:22:05.663769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148886+0000","last_change":"2023-12-14T18:24:40.601995+0000","last_active":"2023-12-14T18:24:41.148886+0000","last_peered":"2023-12-14T18:24:41.148886+0000","last_clean":"2023-12-14T18:24:41.148886+0000","last_became_active":"2023-12-14T18:24:40.601571+0000","last_became_peered":"2023-12-14T18:24:40.601571+0000","last_unstale":"2023-12-14T18:24:41.148886+0000","last_undegraded":"2023-12-14T18:24:41.148886+0000","last_fullsized":"2023-12-14T18:24:41.148886+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:30:58.566301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.756794+0000","last_change":"2023-12-14T18:24:36.168661+0000","last_active":"2023-12-14T18:25:00.756794+0000","last_peered":"2023-12-14T18:25:00.756794+0000","last_clean":"2023-12-14T18:25:00.756794+0000","last_became_active":"2023-12-14T18:24:36.168299+0000","last_became_peered":"2023-12-14T18:24:36.168299+0000","last_unstale":"2023-12-14T18:25:00.756794+0000","last_undegraded":"2023-12-14T18:25:00.756794+0000","last_fullsized":"2023-12-14T18:25:00.756794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:55.288902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939900+0000","last_change":"2023-12-14T18:24:36.578417+0000","last_active":"2023-12-14T18:24:41.939900+0000","last_peered":"2023-12-14T18:24:41.939900+0000","last_clean":"2023-12-14T18:24:41.939900+0000","last_became_active":"2023-12-14T18:24:36.578080+0000","last_became_peered":"2023-12-14T18:24:36.578080+0000","last_unstale":"2023-12-14T18:24:41.939900+0000","last_undegraded":"2023-12-14T18:24:41.939900+0000","last_fullsized":"2023-12-14T18:24:41.939900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:07:18.978037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095510+0000","last_change":"2023-12-14T18:24:40.771866+0000","last_active":"2023-12-14T18:24:41.095510+0000","last_peered":"2023-12-14T18:24:41.095510+0000","last_clean":"2023-12-14T18:24:41.095510+0000","last_became_active":"2023-12-14T18:24:40.771324+0000","last_became_peered":"2023-12-14T18:24:40.771324+0000","last_unstale":"2023-12-14T18:24:41.095510+0000","last_undegraded":"2023-12-14T18:24:41.095510+0000","last_fullsized":"2023-12-14T18:24:41.095510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:20:19.058538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093047+0000","last_change":"2023-12-14T18:24:36.170489+0000","last_active":"2023-12-14T18:24:41.093047+0000","last_peered":"2023-12-14T18:24:41.093047+0000","last_clean":"2023-12-14T18:24:41.093047+0000","last_became_active":"2023-12-14T18:24:36.170161+0000","last_became_peered":"2023-12-14T18:24:36.170161+0000","last_unstale":"2023-12-14T18:24:41.093047+0000","last_undegraded":"2023-12-14T18:24:41.093047+0000","last_fullsized":"2023-12-14T18:24:41.093047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:33:56.690387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940761+0000","last_change":"2023-12-14T18:24:36.591094+0000","last_active":"2023-12-14T18:24:41.940761+0000","last_peered":"2023-12-14T18:24:41.940761+0000","last_clean":"2023-12-14T18:24:41.940761+0000","last_became_active":"2023-12-14T18:24:36.590779+0000","last_became_peered":"2023-12-14T18:24:36.590779+0000","last_unstale":"2023-12-14T18:24:41.940761+0000","last_undegraded":"2023-12-14T18:24:41.940761+0000","last_fullsized":"2023-12-14T18:24:41.940761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:16:00.885373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940628+0000","last_change":"2023-12-14T18:24:40.604125+0000","last_active":"2023-12-14T18:24:41.940628+0000","last_peered":"2023-12-14T18:24:41.940628+0000","last_clean":"2023-12-14T18:24:41.940628+0000","last_became_active":"2023-12-14T18:24:40.603242+0000","last_became_peered":"2023-12-14T18:24:40.603242+0000","last_unstale":"2023-12-14T18:24:41.940628+0000","last_undegraded":"2023-12-14T18:24:41.940628+0000","last_fullsized":"2023-12-14T18:24:41.940628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:52:35.630493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940851+0000","last_change":"2023-12-14T18:24:36.177567+0000","last_active":"2023-12-14T18:24:41.940851+0000","last_peered":"2023-12-14T18:24:41.940851+0000","last_clean":"2023-12-14T18:24:41.940851+0000","last_became_active":"2023-12-14T18:24:36.176949+0000","last_became_peered":"2023-12-14T18:24:36.176949+0000","last_unstale":"2023-12-14T18:24:41.940851+0000","last_undegraded":"2023-12-14T18:24:41.940851+0000","last_fullsized":"2023-12-14T18:24:41.940851+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:13:55.523229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940032+0000","last_change":"2023-12-14T18:24:36.594224+0000","last_active":"2023-12-14T18:24:41.940032+0000","last_peered":"2023-12-14T18:24:41.940032+0000","last_clean":"2023-12-14T18:24:41.940032+0000","last_became_active":"2023-12-14T18:24:36.593859+0000","last_became_peered":"2023-12-14T18:24:36.593859+0000","last_unstale":"2023-12-14T18:24:41.940032+0000","last_undegraded":"2023-12-14T18:24:41.940032+0000","last_fullsized":"2023-12-14T18:24:41.940032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:59:04.044943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940173+0000","last_change":"2023-12-14T18:24:40.771814+0000","last_active":"2023-12-14T18:24:41.940173+0000","last_peered":"2023-12-14T18:24:41.940173+0000","last_clean":"2023-12-14T18:24:41.940173+0000","last_became_active":"2023-12-14T18:24:40.771387+0000","last_became_peered":"2023-12-14T18:24:40.771387+0000","last_unstale":"2023-12-14T18:24:41.940173+0000","last_undegraded":"2023-12-14T18:24:41.940173+0000","last_fullsized":"2023-12-14T18:24:41.940173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:22.744059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148815+0000","last_change":"2023-12-14T18:24:36.588826+0000","last_active":"2023-12-14T18:24:41.148815+0000","last_peered":"2023-12-14T18:24:41.148815+0000","last_clean":"2023-12-14T18:24:41.148815+0000","last_became_active":"2023-12-14T18:24:36.588320+0000","last_became_peered":"2023-12-14T18:24:36.588320+0000","last_unstale":"2023-12-14T18:24:41.148815+0000","last_undegraded":"2023-12-14T18:24:41.148815+0000","last_fullsized":"2023-12-14T18:24:41.148815+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:15:09.573489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:34.832934+0000","last_change":"2023-12-14T18:24:36.173995+0000","last_active":"2023-12-14T18:25:34.832934+0000","last_peered":"2023-12-14T18:25:34.832934+0000","last_clean":"2023-12-14T18:25:34.832934+0000","last_became_active":"2023-12-14T18:24:36.173381+0000","last_became_peered":"2023-12-14T18:24:36.173381+0000","last_unstale":"2023-12-14T18:25:34.832934+0000","last_undegraded":"2023-12-14T18:25:34.832934+0000","last_fullsized":"2023-12-14T18:25:34.832934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:17:33.452268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:42.862743+0000","last_change":"2023-12-14T18:24:40.605099+0000","last_active":"2023-12-14T18:24:42.862743+0000","last_peered":"2023-12-14T18:24:42.862743+0000","last_clean":"2023-12-14T18:24:42.862743+0000","last_became_active":"2023-12-14T18:24:40.604773+0000","last_became_peered":"2023-12-14T18:24:40.604773+0000","last_unstale":"2023-12-14T18:24:42.862743+0000","last_undegraded":"2023-12-14T18:24:42.862743+0000","last_fullsized":"2023-12-14T18:24:42.862743+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:27:44.202301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998989+0000","last_change":"2023-12-14T18:24:36.574993+0000","last_active":"2023-12-14T18:24:40.998989+0000","last_peered":"2023-12-14T18:24:40.998989+0000","last_clean":"2023-12-14T18:24:40.998989+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:24:40.998989+0000","last_undegraded":"2023-12-14T18:24:40.998989+0000","last_fullsized":"2023-12-14T18:24:40.998989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:36:49.384865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767286+0000","last_change":"2023-12-14T18:24:36.170061+0000","last_active":"2023-12-14T18:25:00.767286+0000","last_peered":"2023-12-14T18:25:00.767286+0000","last_clean":"2023-12-14T18:25:00.767286+0000","last_became_active":"2023-12-14T18:24:36.169652+0000","last_became_peered":"2023-12-14T18:24:36.169652+0000","last_unstale":"2023-12-14T18:25:00.767286+0000","last_undegraded":"2023-12-14T18:25:00.767286+0000","last_fullsized":"2023-12-14T18:25:00.767286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:35:33.204173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":344,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941240+0000","last_change":"2023-12-14T18:23:49.076644+0000","last_active":"2023-12-14T18:24:41.941240+0000","last_peered":"2023-12-14T18:24:41.941240+0000","last_clean":"2023-12-14T18:24:41.941240+0000","last_became_active":"2023-12-14T18:23:49.076263+0000","last_became_peered":"2023-12-14T18:23:49.076263+0000","last_unstale":"2023-12-14T18:24:41.941240+0000","last_undegraded":"2023-12-14T18:24:41.941240+0000","last_fullsized":"2023-12-14T18:24:41.941240+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_clean_scrub_stamp":"2023-12-14T18:21:24.715131+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:19:15.132541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149430+0000","last_change":"2023-12-14T18:24:40.772015+0000","last_active":"2023-12-14T18:24:41.149430+0000","last_peered":"2023-12-14T18:24:41.149430+0000","last_clean":"2023-12-14T18:24:41.149430+0000","last_became_active":"2023-12-14T18:24:40.771497+0000","last_became_peered":"2023-12-14T18:24:40.771497+0000","last_unstale":"2023-12-14T18:24:41.149430+0000","last_undegraded":"2023-12-14T18:24:41.149430+0000","last_fullsized":"2023-12-14T18:24:41.149430+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:37:04.368364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939552+0000","last_change":"2023-12-14T18:24:36.584402+0000","last_active":"2023-12-14T18:24:41.939552+0000","last_peered":"2023-12-14T18:24:41.939552+0000","last_clean":"2023-12-14T18:24:41.939552+0000","last_became_active":"2023-12-14T18:24:36.584177+0000","last_became_peered":"2023-12-14T18:24:36.584177+0000","last_unstale":"2023-12-14T18:24:41.939552+0000","last_undegraded":"2023-12-14T18:24:41.939552+0000","last_fullsized":"2023-12-14T18:24:41.939552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:38:45.449137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767235+0000","last_change":"2023-12-14T18:24:36.175017+0000","last_active":"2023-12-14T18:25:00.767235+0000","last_peered":"2023-12-14T18:25:00.767235+0000","last_clean":"2023-12-14T18:25:00.767235+0000","last_became_active":"2023-12-14T18:24:36.174639+0000","last_became_peered":"2023-12-14T18:24:36.174639+0000","last_unstale":"2023-12-14T18:25:00.767235+0000","last_undegraded":"2023-12-14T18:25:00.767235+0000","last_fullsized":"2023-12-14T18:25:00.767235+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:56:02.780313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939585+0000","last_change":"2023-12-14T18:24:40.596043+0000","last_active":"2023-12-14T18:24:41.939585+0000","last_peered":"2023-12-14T18:24:41.939585+0000","last_clean":"2023-12-14T18:24:41.939585+0000","last_became_active":"2023-12-14T18:24:40.595603+0000","last_became_peered":"2023-12-14T18:24:40.595603+0000","last_unstale":"2023-12-14T18:24:41.939585+0000","last_undegraded":"2023-12-14T18:24:41.939585+0000","last_fullsized":"2023-12-14T18:24:41.939585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:09:47.437024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.038437+0000","last_change":"2023-12-14T18:24:36.177484+0000","last_active":"2023-12-14T18:25:00.038437+0000","last_peered":"2023-12-14T18:25:00.038437+0000","last_clean":"2023-12-14T18:25:00.038437+0000","last_became_active":"2023-12-14T18:24:36.176592+0000","last_became_peered":"2023-12-14T18:24:36.176592+0000","last_unstale":"2023-12-14T18:25:00.038437+0000","last_undegraded":"2023-12-14T18:25:00.038437+0000","last_fullsized":"2023-12-14T18:25:00.038437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:21:55.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940747+0000","last_change":"2023-12-14T18:24:36.585687+0000","last_active":"2023-12-14T18:24:41.940747+0000","last_peered":"2023-12-14T18:24:41.940747+0000","last_clean":"2023-12-14T18:24:41.940747+0000","last_became_active":"2023-12-14T18:24:36.585360+0000","last_became_peered":"2023-12-14T18:24:36.585360+0000","last_unstale":"2023-12-14T18:24:41.940747+0000","last_undegraded":"2023-12-14T18:24:41.940747+0000","last_fullsized":"2023-12-14T18:24:41.940747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:42.737826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998830+0000","last_change":"2023-12-14T18:24:40.771747+0000","last_active":"2023-12-14T18:24:40.998830+0000","last_peered":"2023-12-14T18:24:40.998830+0000","last_clean":"2023-12-14T18:24:40.998830+0000","last_became_active":"2023-12-14T18:24:40.771310+0000","last_became_peered":"2023-12-14T18:24:40.771310+0000","last_unstale":"2023-12-14T18:24:40.998830+0000","last_undegraded":"2023-12-14T18:24:40.998830+0000","last_fullsized":"2023-12-14T18:24:40.998830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:52.843438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:06.093093+0000","last_change":"2023-12-14T18:24:36.171833+0000","last_active":"2023-12-14T18:25:06.093093+0000","last_peered":"2023-12-14T18:25:06.093093+0000","last_clean":"2023-12-14T18:25:06.093093+0000","last_became_active":"2023-12-14T18:24:36.171512+0000","last_became_peered":"2023-12-14T18:24:36.171512+0000","last_unstale":"2023-12-14T18:25:06.093093+0000","last_undegraded":"2023-12-14T18:25:06.093093+0000","last_fullsized":"2023-12-14T18:25:06.093093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:56.553040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999452+0000","last_change":"2023-12-14T18:24:36.594938+0000","last_active":"2023-12-14T18:24:40.999452+0000","last_peered":"2023-12-14T18:24:40.999452+0000","last_clean":"2023-12-14T18:24:40.999452+0000","last_became_active":"2023-12-14T18:24:36.594573+0000","last_became_peered":"2023-12-14T18:24:36.594573+0000","last_unstale":"2023-12-14T18:24:40.999452+0000","last_undegraded":"2023-12-14T18:24:40.999452+0000","last_fullsized":"2023-12-14T18:24:40.999452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:18:30.843187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000512+0000","last_change":"2023-12-14T18:24:40.601146+0000","last_active":"2023-12-14T18:24:41.000512+0000","last_peered":"2023-12-14T18:24:41.000512+0000","last_clean":"2023-12-14T18:24:41.000512+0000","last_became_active":"2023-12-14T18:24:40.600745+0000","last_became_peered":"2023-12-14T18:24:40.600745+0000","last_unstale":"2023-12-14T18:24:41.000512+0000","last_undegraded":"2023-12-14T18:24:41.000512+0000","last_fullsized":"2023-12-14T18:24:41.000512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:48:20.003480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039339+0000","last_change":"2023-12-14T18:24:36.175880+0000","last_active":"2023-12-14T18:25:00.039339+0000","last_peered":"2023-12-14T18:25:00.039339+0000","last_clean":"2023-12-14T18:25:00.039339+0000","last_became_active":"2023-12-14T18:24:36.175514+0000","last_became_peered":"2023-12-14T18:24:36.175514+0000","last_unstale":"2023-12-14T18:25:00.039339+0000","last_undegraded":"2023-12-14T18:25:00.039339+0000","last_fullsized":"2023-12-14T18:25:00.039339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:00:40.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939608+0000","last_change":"2023-12-14T18:24:36.577062+0000","last_active":"2023-12-14T18:24:41.939608+0000","last_peered":"2023-12-14T18:24:41.939608+0000","last_clean":"2023-12-14T18:24:41.939608+0000","last_became_active":"2023-12-14T18:24:36.576733+0000","last_became_peered":"2023-12-14T18:24:36.576733+0000","last_unstale":"2023-12-14T18:24:41.939608+0000","last_undegraded":"2023-12-14T18:24:41.939608+0000","last_fullsized":"2023-12-14T18:24:41.939608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:12:08.207474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940397+0000","last_change":"2023-12-14T18:24:40.772752+0000","last_active":"2023-12-14T18:24:41.940397+0000","last_peered":"2023-12-14T18:24:41.940397+0000","last_clean":"2023-12-14T18:24:41.940397+0000","last_became_active":"2023-12-14T18:24:40.772168+0000","last_became_peered":"2023-12-14T18:24:40.772168+0000","last_unstale":"2023-12-14T18:24:41.940397+0000","last_undegraded":"2023-12-14T18:24:41.940397+0000","last_fullsized":"2023-12-14T18:24:41.940397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:02:24.129455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.038773+0000","last_change":"2023-12-14T18:24:36.178885+0000","last_active":"2023-12-14T18:25:00.038773+0000","last_peered":"2023-12-14T18:25:00.038773+0000","last_clean":"2023-12-14T18:25:00.038773+0000","last_became_active":"2023-12-14T18:24:36.178386+0000","last_became_peered":"2023-12-14T18:24:36.178386+0000","last_unstale":"2023-12-14T18:25:00.038773+0000","last_undegraded":"2023-12-14T18:25:00.038773+0000","last_fullsized":"2023-12-14T18:25:00.038773+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:37:16.134801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095247+0000","last_change":"2023-12-14T18:24:36.590037+0000","last_active":"2023-12-14T18:24:41.095247+0000","last_peered":"2023-12-14T18:24:41.095247+0000","last_clean":"2023-12-14T18:24:41.095247+0000","last_became_active":"2023-12-14T18:24:36.589659+0000","last_became_peered":"2023-12-14T18:24:36.589659+0000","last_unstale":"2023-12-14T18:24:41.095247+0000","last_undegraded":"2023-12-14T18:24:41.095247+0000","last_fullsized":"2023-12-14T18:24:41.095247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:13:57.949133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093418+0000","last_change":"2023-12-14T18:24:40.602483+0000","last_active":"2023-12-14T18:24:41.093418+0000","last_peered":"2023-12-14T18:24:41.093418+0000","last_clean":"2023-12-14T18:24:41.093418+0000","last_became_active":"2023-12-14T18:24:40.601989+0000","last_became_peered":"2023-12-14T18:24:40.601989+0000","last_unstale":"2023-12-14T18:24:41.093418+0000","last_undegraded":"2023-12-14T18:24:41.093418+0000","last_fullsized":"2023-12-14T18:24:41.093418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:09.202695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.037079+0000","last_change":"2023-12-14T18:24:36.173350+0000","last_active":"2023-12-14T18:25:00.037079+0000","last_peered":"2023-12-14T18:25:00.037079+0000","last_clean":"2023-12-14T18:25:00.037079+0000","last_became_active":"2023-12-14T18:24:36.172840+0000","last_became_peered":"2023-12-14T18:24:36.172840+0000","last_unstale":"2023-12-14T18:25:00.037079+0000","last_undegraded":"2023-12-14T18:25:00.037079+0000","last_fullsized":"2023-12-14T18:25:00.037079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:17:31.439315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939457+0000","last_change":"2023-12-14T18:24:36.592104+0000","last_active":"2023-12-14T18:24:41.939457+0000","last_peered":"2023-12-14T18:24:41.939457+0000","last_clean":"2023-12-14T18:24:41.939457+0000","last_became_active":"2023-12-14T18:24:36.591568+0000","last_became_peered":"2023-12-14T18:24:36.591568+0000","last_unstale":"2023-12-14T18:24:41.939457+0000","last_undegraded":"2023-12-14T18:24:41.939457+0000","last_fullsized":"2023-12-14T18:24:41.939457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:30.952113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940589+0000","last_change":"2023-12-14T18:24:40.596043+0000","last_active":"2023-12-14T18:24:41.940589+0000","last_peered":"2023-12-14T18:24:41.940589+0000","last_clean":"2023-12-14T18:24:41.940589+0000","last_became_active":"2023-12-14T18:24:40.595520+0000","last_became_peered":"2023-12-14T18:24:40.595520+0000","last_unstale":"2023-12-14T18:24:41.940589+0000","last_undegraded":"2023-12-14T18:24:41.940589+0000","last_fullsized":"2023-12-14T18:24:41.940589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:45:44.615227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000728+0000","last_change":"2023-12-14T18:24:36.174184+0000","last_active":"2023-12-14T18:24:41.000728+0000","last_peered":"2023-12-14T18:24:41.000728+0000","last_clean":"2023-12-14T18:24:41.000728+0000","last_became_active":"2023-12-14T18:24:36.173850+0000","last_became_peered":"2023-12-14T18:24:36.173850+0000","last_unstale":"2023-12-14T18:24:41.000728+0000","last_undegraded":"2023-12-14T18:24:41.000728+0000","last_fullsized":"2023-12-14T18:24:41.000728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:49:58.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999206+0000","last_change":"2023-12-14T18:24:36.587777+0000","last_active":"2023-12-14T18:24:40.999206+0000","last_peered":"2023-12-14T18:24:40.999206+0000","last_clean":"2023-12-14T18:24:40.999206+0000","last_became_active":"2023-12-14T18:24:36.587317+0000","last_became_peered":"2023-12-14T18:24:36.587317+0000","last_unstale":"2023-12-14T18:24:40.999206+0000","last_undegraded":"2023-12-14T18:24:40.999206+0000","last_fullsized":"2023-12-14T18:24:40.999206+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:59:49.931215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940400+0000","last_change":"2023-12-14T18:24:40.771913+0000","last_active":"2023-12-14T18:24:41.940400+0000","last_peered":"2023-12-14T18:24:41.940400+0000","last_clean":"2023-12-14T18:24:41.940400+0000","last_became_active":"2023-12-14T18:24:40.771425+0000","last_became_peered":"2023-12-14T18:24:40.771425+0000","last_unstale":"2023-12-14T18:24:41.940400+0000","last_undegraded":"2023-12-14T18:24:41.940400+0000","last_fullsized":"2023-12-14T18:24:41.940400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:17:43.378002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039457+0000","last_change":"2023-12-14T18:24:36.176801+0000","last_active":"2023-12-14T18:25:00.039457+0000","last_peered":"2023-12-14T18:25:00.039457+0000","last_clean":"2023-12-14T18:25:00.039457+0000","last_became_active":"2023-12-14T18:24:36.176001+0000","last_became_peered":"2023-12-14T18:24:36.176001+0000","last_unstale":"2023-12-14T18:25:00.039457+0000","last_undegraded":"2023-12-14T18:25:00.039457+0000","last_fullsized":"2023-12-14T18:25:00.039457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:16:31.583481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940941+0000","last_change":"2023-12-14T18:24:36.592898+0000","last_active":"2023-12-14T18:24:41.940941+0000","last_peered":"2023-12-14T18:24:41.940941+0000","last_clean":"2023-12-14T18:24:41.940941+0000","last_became_active":"2023-12-14T18:24:36.586650+0000","last_became_peered":"2023-12-14T18:24:36.586650+0000","last_unstale":"2023-12-14T18:24:41.940941+0000","last_undegraded":"2023-12-14T18:24:41.940941+0000","last_fullsized":"2023-12-14T18:24:41.940941+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:39:25.660709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000369+0000","last_change":"2023-12-14T18:24:40.599181+0000","last_active":"2023-12-14T18:24:41.000369+0000","last_peered":"2023-12-14T18:24:41.000369+0000","last_clean":"2023-12-14T18:24:41.000369+0000","last_became_active":"2023-12-14T18:24:40.598785+0000","last_became_peered":"2023-12-14T18:24:40.598785+0000","last_unstale":"2023-12-14T18:24:41.000369+0000","last_undegraded":"2023-12-14T18:24:41.000369+0000","last_fullsized":"2023-12-14T18:24:41.000369+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:11:55.342182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767511+0000","last_change":"2023-12-14T18:24:36.173644+0000","last_active":"2023-12-14T18:25:00.767511+0000","last_peered":"2023-12-14T18:25:00.767511+0000","last_clean":"2023-12-14T18:25:00.767511+0000","last_became_active":"2023-12-14T18:24:36.173059+0000","last_became_peered":"2023-12-14T18:24:36.173059+0000","last_unstale":"2023-12-14T18:25:00.767511+0000","last_undegraded":"2023-12-14T18:25:00.767511+0000","last_fullsized":"2023-12-14T18:25:00.767511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:11:21.509881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941424+0000","last_change":"2023-12-14T18:24:36.588761+0000","last_active":"2023-12-14T18:24:41.941424+0000","last_peered":"2023-12-14T18:24:41.941424+0000","last_clean":"2023-12-14T18:24:41.941424+0000","last_became_active":"2023-12-14T18:24:36.586021+0000","last_became_peered":"2023-12-14T18:24:36.586021+0000","last_unstale":"2023-12-14T18:24:41.941424+0000","last_undegraded":"2023-12-14T18:24:41.941424+0000","last_fullsized":"2023-12-14T18:24:41.941424+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:23:36.358426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939203+0000","last_change":"2023-12-14T18:24:40.607286+0000","last_active":"2023-12-14T18:24:41.939203+0000","last_peered":"2023-12-14T18:24:41.939203+0000","last_clean":"2023-12-14T18:24:41.939203+0000","last_became_active":"2023-12-14T18:24:40.606925+0000","last_became_peered":"2023-12-14T18:24:40.606925+0000","last_unstale":"2023-12-14T18:24:41.939203+0000","last_undegraded":"2023-12-14T18:24:41.939203+0000","last_fullsized":"2023-12-14T18:24:41.939203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:47:39.360084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.149531+0000","last_change":"2023-12-14T18:24:36.174467+0000","last_active":"2023-12-14T18:24:41.149531+0000","last_peered":"2023-12-14T18:24:41.149531+0000","last_clean":"2023-12-14T18:24:41.149531+0000","last_became_active":"2023-12-14T18:24:36.173920+0000","last_became_peered":"2023-12-14T18:24:36.173920+0000","last_unstale":"2023-12-14T18:24:41.149531+0000","last_undegraded":"2023-12-14T18:24:41.149531+0000","last_fullsized":"2023-12-14T18:24:41.149531+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:36:25.392784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940237+0000","last_change":"2023-12-14T18:24:36.597537+0000","last_active":"2023-12-14T18:24:41.940237+0000","last_peered":"2023-12-14T18:24:41.940237+0000","last_clean":"2023-12-14T18:24:41.940237+0000","last_became_active":"2023-12-14T18:24:36.594599+0000","last_became_peered":"2023-12-14T18:24:36.594599+0000","last_unstale":"2023-12-14T18:24:41.940237+0000","last_undegraded":"2023-12-14T18:24:41.940237+0000","last_fullsized":"2023-12-14T18:24:41.940237+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:00.181838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148562+0000","last_change":"2023-12-14T18:24:40.597990+0000","last_active":"2023-12-14T18:24:41.148562+0000","last_peered":"2023-12-14T18:24:41.148562+0000","last_clean":"2023-12-14T18:24:41.148562+0000","last_became_active":"2023-12-14T18:24:40.597568+0000","last_became_peered":"2023-12-14T18:24:40.597568+0000","last_unstale":"2023-12-14T18:24:41.148562+0000","last_undegraded":"2023-12-14T18:24:41.148562+0000","last_fullsized":"2023-12-14T18:24:41.148562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:31:38.963162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940025+0000","last_change":"2023-12-14T18:24:36.173141+0000","last_active":"2023-12-14T18:24:41.940025+0000","last_peered":"2023-12-14T18:24:41.940025+0000","last_clean":"2023-12-14T18:24:41.940025+0000","last_became_active":"2023-12-14T18:24:36.172811+0000","last_became_peered":"2023-12-14T18:24:36.172811+0000","last_unstale":"2023-12-14T18:24:41.940025+0000","last_undegraded":"2023-12-14T18:24:41.940025+0000","last_fullsized":"2023-12-14T18:24:41.940025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:01:41.375107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941679+0000","last_change":"2023-12-14T18:24:36.587333+0000","last_active":"2023-12-14T18:24:41.941679+0000","last_peered":"2023-12-14T18:24:41.941679+0000","last_clean":"2023-12-14T18:24:41.941679+0000","last_became_active":"2023-12-14T18:24:36.586914+0000","last_became_peered":"2023-12-14T18:24:36.586914+0000","last_unstale":"2023-12-14T18:24:41.941679+0000","last_undegraded":"2023-12-14T18:24:41.941679+0000","last_fullsized":"2023-12-14T18:24:41.941679+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:44:34.232245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939625+0000","last_change":"2023-12-14T18:24:36.591896+0000","last_active":"2023-12-14T18:24:41.939625+0000","last_peered":"2023-12-14T18:24:41.939625+0000","last_clean":"2023-12-14T18:24:41.939625+0000","last_became_active":"2023-12-14T18:24:36.591097+0000","last_became_peered":"2023-12-14T18:24:36.591097+0000","last_unstale":"2023-12-14T18:24:41.939625+0000","last_undegraded":"2023-12-14T18:24:41.939625+0000","last_fullsized":"2023-12-14T18:24:41.939625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:14:56.435755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000778+0000","last_change":"2023-12-14T18:24:36.581688+0000","last_active":"2023-12-14T18:24:41.000778+0000","last_peered":"2023-12-14T18:24:41.000778+0000","last_clean":"2023-12-14T18:24:41.000778+0000","last_became_active":"2023-12-14T18:24:36.580660+0000","last_became_peered":"2023-12-14T18:24:36.580660+0000","last_unstale":"2023-12-14T18:24:41.000778+0000","last_undegraded":"2023-12-14T18:24:41.000778+0000","last_fullsized":"2023-12-14T18:24:41.000778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:13:48.827412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000650+0000","last_change":"2023-12-14T18:24:36.583025+0000","last_active":"2023-12-14T18:24:41.000650+0000","last_peered":"2023-12-14T18:24:41.000650+0000","last_clean":"2023-12-14T18:24:41.000650+0000","last_became_active":"2023-12-14T18:24:36.582510+0000","last_became_peered":"2023-12-14T18:24:36.582510+0000","last_unstale":"2023-12-14T18:24:41.000650+0000","last_undegraded":"2023-12-14T18:24:41.000650+0000","last_fullsized":"2023-12-14T18:24:41.000650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:21:38.503753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939373+0000","last_change":"2023-12-14T18:24:36.579383+0000","last_active":"2023-12-14T18:24:41.939373+0000","last_peered":"2023-12-14T18:24:41.939373+0000","last_clean":"2023-12-14T18:24:41.939373+0000","last_became_active":"2023-12-14T18:24:36.579037+0000","last_became_peered":"2023-12-14T18:24:36.579037+0000","last_unstale":"2023-12-14T18:24:41.939373+0000","last_undegraded":"2023-12-14T18:24:41.939373+0000","last_fullsized":"2023-12-14T18:24:41.939373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:13:32.810036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940749+0000","last_change":"2023-12-14T18:24:36.582940+0000","last_active":"2023-12-14T18:24:41.940749+0000","last_peered":"2023-12-14T18:24:41.940749+0000","last_clean":"2023-12-14T18:24:41.940749+0000","last_became_active":"2023-12-14T18:24:36.582645+0000","last_became_peered":"2023-12-14T18:24:36.582645+0000","last_unstale":"2023-12-14T18:24:41.940749+0000","last_undegraded":"2023-12-14T18:24:41.940749+0000","last_fullsized":"2023-12-14T18:24:41.940749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:43:46.761797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940015+0000","last_change":"2023-12-14T18:24:36.578613+0000","last_active":"2023-12-14T18:24:41.940015+0000","last_peered":"2023-12-14T18:24:41.940015+0000","last_clean":"2023-12-14T18:24:41.940015+0000","last_became_active":"2023-12-14T18:24:36.578287+0000","last_became_peered":"2023-12-14T18:24:36.578287+0000","last_unstale":"2023-12-14T18:24:41.940015+0000","last_undegraded":"2023-12-14T18:24:41.940015+0000","last_fullsized":"2023-12-14T18:24:41.940015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:05:43.072668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001258+0000","last_change":"2023-12-14T18:24:36.575078+0000","last_active":"2023-12-14T18:24:41.001258+0000","last_peered":"2023-12-14T18:24:41.001258+0000","last_clean":"2023-12-14T18:24:41.001258+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:24:41.001258+0000","last_undegraded":"2023-12-14T18:24:41.001258+0000","last_fullsized":"2023-12-14T18:24:41.001258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:55:31.886500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999571+0000","last_change":"2023-12-14T18:24:36.572545+0000","last_active":"2023-12-14T18:24:40.999571+0000","last_peered":"2023-12-14T18:24:40.999571+0000","last_clean":"2023-12-14T18:24:40.999571+0000","last_became_active":"2023-12-14T18:24:36.572235+0000","last_became_peered":"2023-12-14T18:24:36.572235+0000","last_unstale":"2023-12-14T18:24:40.999571+0000","last_undegraded":"2023-12-14T18:24:40.999571+0000","last_fullsized":"2023-12-14T18:24:40.999571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:25:56.081431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.148685+0000","last_change":"2023-12-14T18:24:36.588278+0000","last_active":"2023-12-14T18:24:41.148685+0000","last_peered":"2023-12-14T18:24:41.148685+0000","last_clean":"2023-12-14T18:24:41.148685+0000","last_became_active":"2023-12-14T18:24:36.587605+0000","last_became_peered":"2023-12-14T18:24:36.587605+0000","last_unstale":"2023-12-14T18:24:41.148685+0000","last_undegraded":"2023-12-14T18:24:41.148685+0000","last_fullsized":"2023-12-14T18:24:41.148685+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:32:11.935687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998910+0000","last_change":"2023-12-14T18:24:36.575325+0000","last_active":"2023-12-14T18:24:40.998910+0000","last_peered":"2023-12-14T18:24:40.998910+0000","last_clean":"2023-12-14T18:24:40.998910+0000","last_became_active":"2023-12-14T18:24:36.575016+0000","last_became_peered":"2023-12-14T18:24:36.575016+0000","last_unstale":"2023-12-14T18:24:40.998910+0000","last_undegraded":"2023-12-14T18:24:40.998910+0000","last_fullsized":"2023-12-14T18:24:40.998910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:35:26.394701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000615+0000","last_change":"2023-12-14T18:24:36.585348+0000","last_active":"2023-12-14T18:24:41.000615+0000","last_peered":"2023-12-14T18:24:41.000615+0000","last_clean":"2023-12-14T18:24:41.000615+0000","last_became_active":"2023-12-14T18:24:36.584995+0000","last_became_peered":"2023-12-14T18:24:36.584995+0000","last_unstale":"2023-12-14T18:24:41.000615+0000","last_undegraded":"2023-12-14T18:24:41.000615+0000","last_fullsized":"2023-12-14T18:24:41.000615+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:22:10.788069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939979+0000","last_change":"2023-12-14T18:24:36.583571+0000","last_active":"2023-12-14T18:24:41.939979+0000","last_peered":"2023-12-14T18:24:41.939979+0000","last_clean":"2023-12-14T18:24:41.939979+0000","last_became_active":"2023-12-14T18:24:36.583266+0000","last_became_peered":"2023-12-14T18:24:36.583266+0000","last_unstale":"2023-12-14T18:24:41.939979+0000","last_undegraded":"2023-12-14T18:24:41.939979+0000","last_fullsized":"2023-12-14T18:24:41.939979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:09:22.877080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939741+0000","last_change":"2023-12-14T18:24:36.579754+0000","last_active":"2023-12-14T18:24:41.939741+0000","last_peered":"2023-12-14T18:24:41.939741+0000","last_clean":"2023-12-14T18:24:41.939741+0000","last_became_active":"2023-12-14T18:24:36.579337+0000","last_became_peered":"2023-12-14T18:24:36.579337+0000","last_unstale":"2023-12-14T18:24:41.939741+0000","last_undegraded":"2023-12-14T18:24:41.939741+0000","last_fullsized":"2023-12-14T18:24:41.939741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:48:45.306297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212533,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64740,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":236,"num_read_kb":189,"num_write":153,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547222,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6088,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739064,"statfs":{"total":95995035648,"available":95988801536,"internally_reserved":0,"allocated":917504,"data_stored":458640,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.379},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.436}]},{"osd":5,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":9,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.753},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.67100000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.78100000000000003}]}]},{"osd":10,"up_from":67,"seq":287762808858,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":1056768,"data_stored":1106365,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.6690000000000005,"5min":8.6690000000000005,"15min":8.6690000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":9.5459999999999994,"5min":9.5459999999999994,"15min":9.5459999999999994},"last":0.33300000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":9.4380000000000006,"5min":9.4380000000000006,"15min":9.4380000000000006},"last":0.498},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.9470000000000001,"5min":5.9470000000000001,"15min":5.9470000000000001},"last":0.379},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":6.1470000000000002,"5min":6.1470000000000002,"15min":6.1470000000000002},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":9.4830000000000005,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":0.439},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.48399999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.54100000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":6.0309999999999997,"5min":6.0309999999999997,"15min":6.0309999999999997},"last":0.52400000000000002}]}]},{"osd":9,"up_from":62,"seq":266287972380,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6236,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738916,"statfs":{"total":95995035648,"available":95988649984,"internally_reserved":0,"allocated":937984,"data_stored":462972,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42199999999999999,"15min":0.42999999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53500000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.27200000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.73999999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60999999999999999,"15min":0.62},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.83399999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61199999999999999,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.438}]},{"osd":3,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.47199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59199999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.879,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72999999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.45900000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80700000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.749},"min":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84799999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.61099999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.81699999999999995,"5min":1.214,"15min":1.214},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.47199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79600000000000004,"5min":1.071,"15min":1.071},"last":0.622}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]},{"osd":8,"up_from":55,"seq":236223201313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6356,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738796,"statfs":{"total":95995035648,"available":95988527104,"internally_reserved":0,"allocated":929792,"data_stored":458465,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.501},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.372},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.41299999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47599999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.628},{"interface":"front","average":{"1min":0.754,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71699999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":7,"up_from":48,"seq":206158430243,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6468,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738684,"statfs":{"total":95995035648,"available":95988412416,"internally_reserved":0,"allocated":978944,"data_stored":1054277,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.434}]},{"osd":1,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.47599999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.59499999999999997}]},{"osd":4,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.53600000000000003,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.7830000000000004,"5min":5.7830000000000004,"15min":5.7830000000000004},"last":0.433},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.44900000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.52000000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77500000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.63700000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.58999999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82699999999999996,"5min":0.879,"15min":0.879},"last":0.71999999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.74199999999999999,"15min":0.74199999999999999},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.64700000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.68000000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]}]},{"osd":6,"up_from":43,"seq":184683593766,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6340,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738812,"statfs":{"total":95995035648,"available":95988543488,"internally_reserved":0,"allocated":913408,"data_stored":454052,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38500000000000001},"min":{"1min":0.23400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.253,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70399999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.745,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70899999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.46400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.624,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56499999999999995}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.65200000000000002,"15min":0.65700000000000003},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79200000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.501},{"interface":"front","average":{"1min":0.627,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79800000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47199999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.60199999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]},{"osd":1,"up_from":13,"seq":55834574905,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7364,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737788,"statfs":{"total":95995035648,"available":95987494912,"internally_reserved":0,"allocated":978944,"data_stored":514798,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.46200000000000002},"min":{"1min":0.36099999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.51700000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:24:30 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.498,"15min":0.5},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.50900000000000001,"15min":0.51200000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"last":0.751}]},{"osd":3,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47799999999999998,"15min":0.46500000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.59099999999999997}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.316,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.62},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.40300000000000002},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77600000000000002,"5min":1.542,"15min":1.542},"last":0.67600000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53800000000000003,"15min":0.51900000000000002},"min":{"1min":0.442,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.82599999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.60699999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83299999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.627}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.501,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.86199999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52900000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.71499999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.92700000000000005}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7196,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737956,"statfs":{"total":95995035648,"available":95987666944,"internally_reserved":0,"allocated":937984,"data_stored":455440,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.52500000000000002,"15min":0.497},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54000000000000004,"15min":0.5},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.79000000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.27300000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35499999999999998},"min":{"1min":0.26700000000000002,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.38,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51200000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71799999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.51300000000000001,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.626},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.40400000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.376},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.74099999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76500000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.63900000000000001,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":7.2290000000000001,"5min":7.2290000000000001,"15min":7.2290000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.67300000000000004,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.37,"15min":0.37},"max":{"1min":7.3390000000000004,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.47899999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58499999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70899999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7452,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737700,"statfs":{"total":95995035648,"available":95987404800,"internally_reserved":0,"allocated":1003520,"data_stored":516490,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.434},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.64200000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.44900000000000001},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.499,"15min":0.47099999999999997},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.51500000000000001,"15min":0.48599999999999999},"min":{"1min":0.39500000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.85599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.46600000000000003},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76100000000000001,"5min":1.179,"15min":1.179},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.39700000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.437,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.72199999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67300000000000004}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72899999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.76900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":3,"up_from":24,"seq":103079215154,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":925696,"data_stored":454246,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.32800000000000001},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.503},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.248,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.497,"15min":0.47499999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.626}]},{"osd":2,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":1.214,"15min":1.214},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.45300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64100000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66400000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.497,"15min":0.499},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":0.755},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:24:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.628},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.85099999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79900000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.63700000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63700000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986223,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":925696,"data_stored":458654,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48799999999999999,"15min":0.48099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.622},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.48399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.45000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.76500000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44400000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.36899999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.503,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59799999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.51900000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.84499999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822698,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":991232,"data_stored":1050244,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.374},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.626,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.57799999999999996}]},{"osd":1,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.501,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.625},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.63800000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.254,"15min":0.254},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.32800000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.628,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.49099999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.68000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.68000000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.61499999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.997,"15min":0.997},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.40600000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.65000000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82299999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.68899999999999995}]},{"osd":9,"last update":"Thu Dec 14 18:24:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.90700000000000003,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"last":0.71499999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.74299999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:25:37.420 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph pg dump --format=json 2023-12-14T18:25:37.763 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:38.184 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-14T18:25:38.184 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-12-14T18:25:38.184 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: osdmap e85: 12 total, 12 up, 12 in 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:25:38.185 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:25:38.186 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:37 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: osdmap e85: 12 total, 12 up, 12 in 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:38.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:38.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:38.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:25:38.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:38.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:25:38.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:25:38.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:37 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.6", "id": [8, 11]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.a", "id": [2, 11]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1b", "id": [2, 4]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.10", "id": [5, 9]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.34", "id": [4, 9]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.3", "id": [5, 8]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.c", "id": [9, 10]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.f", "id": [9, 8]}]': finished 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: osdmap e85: 12 total, 12 up, 12 in 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='client.15093 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:25:38.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:25:38.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:25:38.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: from='mgr.14152 172.21.15.93:0/379536810' entity='mgr.x' 2023-12-14T18:25:38.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:25:38.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:25:38.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:37 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:25:38.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.775+0000 7f6983044700 1 -- 172.21.15.93:0/3655185335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c1005b0 msgr2=0x7f697c100980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:38.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.775+0000 7f6983044700 1 --2- 172.21.15.93:0/3655185335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c1005b0 0x7f697c100980 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f69700099c0 tx=0x7f6970009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.775+0000 7f6983044700 1 -- 172.21.15.93:0/3655185335 shutdown_connections 2023-12-14T18:25:38.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.775+0000 7f6983044700 1 --2- 172.21.15.93:0/3655185335 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c1005b0 0x7f697c100980 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.775+0000 7f6983044700 1 --2- 172.21.15.93:0/3655185335 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f697c105650 0x7f697c10d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.775+0000 7f6983044700 1 --2- 172.21.15.93:0/3655185335 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697c100f50 0x7f697c104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.775+0000 7f6983044700 1 -- 172.21.15.93:0/3655185335 >> 172.21.15.93:0/3655185335 conn(0x7f697c077110 msgr2=0x7f697c077510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:38.779 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.775+0000 7f6983044700 1 -- 172.21.15.93:0/3655185335 shutdown_connections 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.775+0000 7f6983044700 1 -- 172.21.15.93:0/3655185335 wait complete. 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f6983044700 1 Processor -- start 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f6983044700 1 -- start start 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f6983044700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697c1005b0 0x7f697c196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f6983044700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f697c100f50 0x7f697c1969c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f6983044700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c105650 0x7f697c19ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f6983044700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c19b440 con 0x7f697c105650 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f6983044700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c19b5b0 con 0x7f697c100f50 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f6983044700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c19b770 con 0x7f697c1005b0 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f69815e1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c105650 0x7f697c19ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f69815e1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c105650 0x7f697c19ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47670/0 (socket says 172.21.15.93:47670) 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f69815e1700 1 -- 172.21.15.93:0/861475159 learned_addr learned my addr 172.21.15.93:0/861475159 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f6980de0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697c1005b0 0x7f697c196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f697bfff700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f697c100f50 0x7f697c1969c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:38.780 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f69815e1700 1 -- 172.21.15.93:0/861475159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697c1005b0 msgr2=0x7f697c196480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f69815e1700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697c1005b0 0x7f697c196480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f69815e1700 1 -- 172.21.15.93:0/861475159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f697c100f50 msgr2=0x7f697c1969c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.776+0000 7f69815e1700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f697c100f50 0x7f697c1969c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.777+0000 7f69815e1700 1 -- 172.21.15.93:0/861475159 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69700095f0 con 0x7f697c105650 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.777+0000 7f69815e1700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c105650 0x7f697c19ae30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f697400b970 tx=0x7f697400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.777+0000 7f6979ffb700 1 -- 172.21.15.93:0/861475159 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f697400d860 con 0x7f697c105650 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.777+0000 7f6979ffb700 1 -- 172.21.15.93:0/861475159 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f697400de80 con 0x7f697c105650 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.777+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c110460 con 0x7f697c105650 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.777+0000 7f6979ffb700 1 -- 172.21.15.93:0/861475159 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6974017400 con 0x7f697c105650 2023-12-14T18:25:38.781 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.777+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c1102d0 con 0x7f697c105650 2023-12-14T18:25:38.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.778+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c1105f0 con 0x7f697c105650 2023-12-14T18:25:38.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.779+0000 7f6979ffb700 1 -- 172.21.15.93:0/861475159 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f697400e630 con 0x7f697c105650 2023-12-14T18:25:38.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.779+0000 7f6979ffb700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f696c0a1260 0x7f696c0a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:38.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.779+0000 7f6980de0700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f696c0a1260 0x7f696c0a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:38.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.780+0000 7f6979ffb700 1 -- 172.21.15.93:0/861475159 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6974006e40 con 0x7f697c105650 2023-12-14T18:25:38.782 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.780+0000 7f6980de0700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f696c0a1260 0x7f696c0a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6970009f50 tx=0x7f6970005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:38.785 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.784+0000 7f6979ffb700 1 -- 172.21.15.93:0/861475159 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6974095c40 con 0x7f697c105650 2023-12-14T18:25:38.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.972+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f697c0fe5e0 con 0x7f696c0a1260 2023-12-14T18:25:38.993 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.983+0000 7f6979ffb700 1 -- 172.21.15.93:0/861475159 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+423751 (secure 0 0 0) 0x7f697c0fe5e0 con 0x7f696c0a1260 2023-12-14T18:25:38.997 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:39.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f696c0a1260 msgr2=0x7f696c0a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:39.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f696c0a1260 0x7f696c0a3710 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6970009f50 tx=0x7f6970005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:39.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c105650 msgr2=0x7f697c19ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:39.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c105650 0x7f697c19ae30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f697400b970 tx=0x7f697400bc80 comp rx=0 tx=0).stop 2023-12-14T18:25:39.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 shutdown_connections 2023-12-14T18:25:39.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f696c0a1260 0x7f696c0a3710 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:39.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f697c105650 0x7f697c19ae30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:39.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f697c1005b0 0x7f697c196480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:39.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 --2- 172.21.15.93:0/861475159 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f697c100f50 0x7f697c1969c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:39.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 >> 172.21.15.93:0/861475159 conn(0x7f697c077110 msgr2=0x7f697c0fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:39.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 shutdown_connections 2023-12-14T18:25:39.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:38.986+0000 7f6983044700 1 -- 172.21.15.93:0/861475159 wait complete. 2023-12-14T18:25:39.001 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-12-14T18:25:39.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:38 smithi093 ceph-mon[42070]: osdmap e86: 12 total, 12 up, 12 in 2023-12-14T18:25:39.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:38 smithi093 ceph-mon[42070]: pgmap v257: 129 pgs: 1 activating, 1 peering, 127 active+clean; 640 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-14T18:25:39.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:38 smithi160 ceph-mon[43840]: osdmap e86: 12 total, 12 up, 12 in 2023-12-14T18:25:39.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:38 smithi160 ceph-mon[43840]: pgmap v257: 129 pgs: 1 activating, 1 peering, 127 active+clean; 640 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-14T18:25:39.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:38 smithi097 ceph-mon[43486]: osdmap e86: 12 total, 12 up, 12 in 2023-12-14T18:25:39.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:38 smithi097 ceph-mon[43486]: pgmap v257: 129 pgs: 1 activating, 1 peering, 127 active+clean; 640 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-14T18:25:39.372 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":257,"stamp":"2023-12-14T18:25:37.960842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655354,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":372,"num_read_kb":308,"num_write":336,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79752,"kb_used_data":11240,"kb_used_omap":0,"kb_used_meta":68416,"kb_avail":1124862072,"statfs":{"total":1151940427776,"available":1151858761728,"internally_reserved":0,"allocated":11509760,"data_stored":7454153,"data_compressed":249644,"data_compressed_allocated":335872,"data_compressed_original":2170880,"omap_allocated":0,"internal_metadata":70057984},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":56,"num_read_kb":32,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002073"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939919+0000","last_change":"2023-12-14T18:24:36.587133+0000","last_active":"2023-12-14T18:24:41.939919+0000","last_peered":"2023-12-14T18:24:41.939919+0000","last_clean":"2023-12-14T18:24:41.939919+0000","last_became_active":"2023-12-14T18:24:36.586222+0000","last_became_peered":"2023-12-14T18:24:36.586222+0000","last_unstale":"2023-12-14T18:24:41.939919+0000","last_undegraded":"2023-12-14T18:24:41.939919+0000","last_fullsized":"2023-12-14T18:24:41.939919+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:51:38.922525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939529+0000","last_change":"2023-12-14T18:24:36.589027+0000","last_active":"2023-12-14T18:24:41.939529+0000","last_peered":"2023-12-14T18:24:41.939529+0000","last_clean":"2023-12-14T18:24:41.939529+0000","last_became_active":"2023-12-14T18:24:36.588785+0000","last_became_peered":"2023-12-14T18:24:36.588785+0000","last_unstale":"2023-12-14T18:24:41.939529+0000","last_undegraded":"2023-12-14T18:24:41.939529+0000","last_fullsized":"2023-12-14T18:24:41.939529+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:42.987957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939327+0000","last_change":"2023-12-14T18:24:36.584015+0000","last_active":"2023-12-14T18:24:41.939327+0000","last_peered":"2023-12-14T18:24:41.939327+0000","last_clean":"2023-12-14T18:24:41.939327+0000","last_became_active":"2023-12-14T18:24:36.583735+0000","last_became_peered":"2023-12-14T18:24:36.583735+0000","last_unstale":"2023-12-14T18:24:41.939327+0000","last_undegraded":"2023-12-14T18:24:41.939327+0000","last_fullsized":"2023-12-14T18:24:41.939327+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:20:02.823540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940899+0000","last_change":"2023-12-14T18:24:36.584553+0000","last_active":"2023-12-14T18:24:41.940899+0000","last_peered":"2023-12-14T18:24:41.940899+0000","last_clean":"2023-12-14T18:24:41.940899+0000","last_became_active":"2023-12-14T18:24:36.583827+0000","last_became_peered":"2023-12-14T18:24:36.583827+0000","last_unstale":"2023-12-14T18:24:41.940899+0000","last_undegraded":"2023-12-14T18:24:41.940899+0000","last_fullsized":"2023-12-14T18:24:41.940899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:32.928164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095366+0000","last_change":"2023-12-14T18:24:36.578128+0000","last_active":"2023-12-14T18:24:41.095366+0000","last_peered":"2023-12-14T18:24:41.095366+0000","last_clean":"2023-12-14T18:24:41.095366+0000","last_became_active":"2023-12-14T18:24:36.577784+0000","last_became_peered":"2023-12-14T18:24:36.577784+0000","last_unstale":"2023-12-14T18:24:41.095366+0000","last_undegraded":"2023-12-14T18:24:41.095366+0000","last_fullsized":"2023-12-14T18:24:41.095366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:17.890524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940347+0000","last_change":"2023-12-14T18:24:36.588666+0000","last_active":"2023-12-14T18:24:41.940347+0000","last_peered":"2023-12-14T18:24:41.940347+0000","last_clean":"2023-12-14T18:24:41.940347+0000","last_became_active":"2023-12-14T18:24:36.588291+0000","last_became_peered":"2023-12-14T18:24:36.588291+0000","last_unstale":"2023-12-14T18:24:41.940347+0000","last_undegraded":"2023-12-14T18:24:41.940347+0000","last_fullsized":"2023-12-14T18:24:41.940347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:10:59.591257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844209+0000","last_change":"2023-12-14T18:24:36.586637+0000","last_active":"2023-12-14T18:25:37.844209+0000","last_peered":"2023-12-14T18:25:37.844209+0000","last_clean":"2023-12-14T18:25:37.844209+0000","last_became_active":"2023-12-14T18:24:36.583055+0000","last_became_peered":"2023-12-14T18:24:36.583055+0000","last_unstale":"2023-12-14T18:25:37.844209+0000","last_undegraded":"2023-12-14T18:25:37.844209+0000","last_fullsized":"2023-12-14T18:25:37.844209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:18:17.900284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998916+0000","last_change":"2023-12-14T18:24:36.580206+0000","last_active":"2023-12-14T18:24:40.998916+0000","last_peered":"2023-12-14T18:24:40.998916+0000","last_clean":"2023-12-14T18:24:40.998916+0000","last_became_active":"2023-12-14T18:24:36.579904+0000","last_became_peered":"2023-12-14T18:24:36.579904+0000","last_unstale":"2023-12-14T18:24:40.998916+0000","last_undegraded":"2023-12-14T18:24:40.998916+0000","last_fullsized":"2023-12-14T18:24:40.998916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:15.182919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.845055+0000","last_change":"2023-12-14T18:24:36.589890+0000","last_active":"2023-12-14T18:25:36.845055+0000","last_peered":"2023-12-14T18:25:36.845055+0000","last_clean":"2023-12-14T18:25:36.845055+0000","last_became_active":"2023-12-14T18:24:36.588999+0000","last_became_peered":"2023-12-14T18:24:36.588999+0000","last_unstale":"2023-12-14T18:25:36.845055+0000","last_undegraded":"2023-12-14T18:25:36.845055+0000","last_fullsized":"2023-12-14T18:25:36.845055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:28:51.096009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941211+0000","last_change":"2023-12-14T18:24:36.589565+0000","last_active":"2023-12-14T18:24:41.941211+0000","last_peered":"2023-12-14T18:24:41.941211+0000","last_clean":"2023-12-14T18:24:41.941211+0000","last_became_active":"2023-12-14T18:24:36.588822+0000","last_became_peered":"2023-12-14T18:24:36.588822+0000","last_unstale":"2023-12-14T18:24:41.941211+0000","last_undegraded":"2023-12-14T18:24:41.941211+0000","last_fullsized":"2023-12-14T18:24:41.941211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:20:30.545830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844272+0000","last_change":"2023-12-14T18:24:36.593208+0000","last_active":"2023-12-14T18:25:37.844272+0000","last_peered":"2023-12-14T18:25:37.844272+0000","last_clean":"2023-12-14T18:25:37.844272+0000","last_became_active":"2023-12-14T18:24:36.586394+0000","last_became_peered":"2023-12-14T18:24:36.586394+0000","last_unstale":"2023-12-14T18:25:37.844272+0000","last_undegraded":"2023-12-14T18:25:37.844272+0000","last_fullsized":"2023-12-14T18:25:37.844272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:22.984010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940763+0000","last_change":"2023-12-14T18:24:36.594254+0000","last_active":"2023-12-14T18:24:41.940763+0000","last_peered":"2023-12-14T18:24:41.940763+0000","last_clean":"2023-12-14T18:24:41.940763+0000","last_became_active":"2023-12-14T18:24:36.593889+0000","last_became_peered":"2023-12-14T18:24:36.593889+0000","last_unstale":"2023-12-14T18:24:41.940763+0000","last_undegraded":"2023-12-14T18:24:41.940763+0000","last_fullsized":"2023-12-14T18:24:41.940763+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:50.487280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939911+0000","last_change":"2023-12-14T18:24:36.584971+0000","last_active":"2023-12-14T18:24:41.939911+0000","last_peered":"2023-12-14T18:24:41.939911+0000","last_clean":"2023-12-14T18:24:41.939911+0000","last_became_active":"2023-12-14T18:24:36.584695+0000","last_became_peered":"2023-12-14T18:24:36.584695+0000","last_unstale":"2023-12-14T18:24:41.939911+0000","last_undegraded":"2023-12-14T18:24:41.939911+0000","last_fullsized":"2023-12-14T18:24:41.939911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:43:12.153321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939181+0000","last_change":"2023-12-14T18:24:36.589692+0000","last_active":"2023-12-14T18:24:41.939181+0000","last_peered":"2023-12-14T18:24:41.939181+0000","last_clean":"2023-12-14T18:24:41.939181+0000","last_became_active":"2023-12-14T18:24:36.589415+0000","last_became_peered":"2023-12-14T18:24:36.589415+0000","last_unstale":"2023-12-14T18:24:41.939181+0000","last_undegraded":"2023-12-14T18:24:41.939181+0000","last_fullsized":"2023-12-14T18:24:41.939181+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:17:55.715589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.938924+0000","last_change":"2023-12-14T18:24:36.586911+0000","last_active":"2023-12-14T18:24:41.938924+0000","last_peered":"2023-12-14T18:24:41.938924+0000","last_clean":"2023-12-14T18:24:41.938924+0000","last_became_active":"2023-12-14T18:24:36.586600+0000","last_became_peered":"2023-12-14T18:24:36.586600+0000","last_unstale":"2023-12-14T18:24:41.938924+0000","last_undegraded":"2023-12-14T18:24:41.938924+0000","last_fullsized":"2023-12-14T18:24:41.938924+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:00:56.150982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.843717+0000","last_change":"2023-12-14T18:24:36.584879+0000","last_active":"2023-12-14T18:25:36.843717+0000","last_peered":"2023-12-14T18:25:36.843717+0000","last_clean":"2023-12-14T18:25:36.843717+0000","last_became_active":"2023-12-14T18:24:36.584566+0000","last_became_peered":"2023-12-14T18:24:36.584566+0000","last_unstale":"2023-12-14T18:25:36.843717+0000","last_undegraded":"2023-12-14T18:25:36.843717+0000","last_fullsized":"2023-12-14T18:25:36.843717+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:04:55.159541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939646+0000","last_change":"2023-12-14T18:24:36.587420+0000","last_active":"2023-12-14T18:24:41.939646+0000","last_peered":"2023-12-14T18:24:41.939646+0000","last_clean":"2023-12-14T18:24:41.939646+0000","last_became_active":"2023-12-14T18:24:36.586732+0000","last_became_peered":"2023-12-14T18:24:36.586732+0000","last_unstale":"2023-12-14T18:24:41.939646+0000","last_undegraded":"2023-12-14T18:24:41.939646+0000","last_fullsized":"2023-12-14T18:24:41.939646+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:06:51.142371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093222+0000","last_change":"2023-12-14T18:24:36.583412+0000","last_active":"2023-12-14T18:24:41.093222+0000","last_peered":"2023-12-14T18:24:41.093222+0000","last_clean":"2023-12-14T18:24:41.093222+0000","last_became_active":"2023-12-14T18:24:36.583022+0000","last_became_peered":"2023-12-14T18:24:36.583022+0000","last_unstale":"2023-12-14T18:24:41.093222+0000","last_undegraded":"2023-12-14T18:24:41.093222+0000","last_fullsized":"2023-12-14T18:24:41.093222+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:35:26.281488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001176+0000","last_change":"2023-12-14T18:24:36.575232+0000","last_active":"2023-12-14T18:24:41.001176+0000","last_peered":"2023-12-14T18:24:41.001176+0000","last_clean":"2023-12-14T18:24:41.001176+0000","last_became_active":"2023-12-14T18:24:36.574573+0000","last_became_peered":"2023-12-14T18:24:36.574573+0000","last_unstale":"2023-12-14T18:24:41.001176+0000","last_undegraded":"2023-12-14T18:24:41.001176+0000","last_fullsized":"2023-12-14T18:24:41.001176+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:09:44.789329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000573+0000","last_change":"2023-12-14T18:24:40.605077+0000","last_active":"2023-12-14T18:24:41.000573+0000","last_peered":"2023-12-14T18:24:41.000573+0000","last_clean":"2023-12-14T18:24:41.000573+0000","last_became_active":"2023-12-14T18:24:40.604646+0000","last_became_peered":"2023-12-14T18:24:40.604646+0000","last_unstale":"2023-12-14T18:24:41.000573+0000","last_undegraded":"2023-12-14T18:24:41.000573+0000","last_fullsized":"2023-12-14T18:24:41.000573+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:46:49.342657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":32,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.768860+0000","last_change":"2023-12-14T18:24:36.177449+0000","last_active":"2023-12-14T18:25:00.768860+0000","last_peered":"2023-12-14T18:25:00.768860+0000","last_clean":"2023-12-14T18:25:00.768860+0000","last_became_active":"2023-12-14T18:24:36.177114+0000","last_became_peered":"2023-12-14T18:24:36.177114+0000","last_unstale":"2023-12-14T18:25:00.768860+0000","last_undegraded":"2023-12-14T18:25:00.768860+0000","last_fullsized":"2023-12-14T18:25:00.768860+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:30.081718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999464+0000","last_change":"2023-12-14T18:24:36.574204+0000","last_active":"2023-12-14T18:24:40.999464+0000","last_peered":"2023-12-14T18:24:40.999464+0000","last_clean":"2023-12-14T18:24:40.999464+0000","last_became_active":"2023-12-14T18:24:36.573923+0000","last_became_peered":"2023-12-14T18:24:36.573923+0000","last_unstale":"2023-12-14T18:24:40.999464+0000","last_undegraded":"2023-12-14T18:24:40.999464+0000","last_fullsized":"2023-12-14T18:24:40.999464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:20:32.637706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939617+0000","last_change":"2023-12-14T18:24:40.600254+0000","last_active":"2023-12-14T18:24:41.939617+0000","last_peered":"2023-12-14T18:24:41.939617+0000","last_clean":"2023-12-14T18:24:41.939617+0000","last_became_active":"2023-12-14T18:24:40.599894+0000","last_became_peered":"2023-12-14T18:24:40.599894+0000","last_unstale":"2023-12-14T18:24:41.939617+0000","last_undegraded":"2023-12-14T18:24:41.939617+0000","last_fullsized":"2023-12-14T18:24:41.939617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:00.116137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999521+0000","last_change":"2023-12-14T18:24:36.170109+0000","last_active":"2023-12-14T18:24:40.999521+0000","last_peered":"2023-12-14T18:24:40.999521+0000","last_clean":"2023-12-14T18:24:40.999521+0000","last_became_active":"2023-12-14T18:24:36.169629+0000","last_became_peered":"2023-12-14T18:24:36.169629+0000","last_unstale":"2023-12-14T18:24:40.999521+0000","last_undegraded":"2023-12-14T18:24:40.999521+0000","last_fullsized":"2023-12-14T18:24:40.999521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:41:16.457493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939537+0000","last_change":"2023-12-14T18:24:36.582757+0000","last_active":"2023-12-14T18:24:41.939537+0000","last_peered":"2023-12-14T18:24:41.939537+0000","last_clean":"2023-12-14T18:24:41.939537+0000","last_became_active":"2023-12-14T18:24:36.582457+0000","last_became_peered":"2023-12-14T18:24:36.582457+0000","last_unstale":"2023-12-14T18:24:41.939537+0000","last_undegraded":"2023-12-14T18:24:41.939537+0000","last_fullsized":"2023-12-14T18:24:41.939537+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:29:05.124428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940384+0000","last_change":"2023-12-14T18:24:40.599695+0000","last_active":"2023-12-14T18:24:41.940384+0000","last_peered":"2023-12-14T18:24:41.940384+0000","last_clean":"2023-12-14T18:24:41.940384+0000","last_became_active":"2023-12-14T18:24:40.599239+0000","last_became_peered":"2023-12-14T18:24:40.599239+0000","last_unstale":"2023-12-14T18:24:41.940384+0000","last_undegraded":"2023-12-14T18:24:41.940384+0000","last_fullsized":"2023-12-14T18:24:41.940384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:40.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.766757+0000","last_change":"2023-12-14T18:24:36.177946+0000","last_active":"2023-12-14T18:25:00.766757+0000","last_peered":"2023-12-14T18:25:00.766757+0000","last_clean":"2023-12-14T18:25:00.766757+0000","last_became_active":"2023-12-14T18:24:36.177480+0000","last_became_peered":"2023-12-14T18:24:36.177480+0000","last_unstale":"2023-12-14T18:25:00.766757+0000","last_undegraded":"2023-12-14T18:25:00.766757+0000","last_fullsized":"2023-12-14T18:25:00.766757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:42:02.447975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.844376+0000","last_change":"2023-12-14T18:24:36.587951+0000","last_active":"2023-12-14T18:25:36.844376+0000","last_peered":"2023-12-14T18:25:36.844376+0000","last_clean":"2023-12-14T18:25:36.844376+0000","last_became_active":"2023-12-14T18:24:36.587539+0000","last_became_peered":"2023-12-14T18:24:36.587539+0000","last_unstale":"2023-12-14T18:25:36.844376+0000","last_undegraded":"2023-12-14T18:25:36.844376+0000","last_fullsized":"2023-12-14T18:25:36.844376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:21:13.030894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941747+0000","last_change":"2023-12-14T18:24:40.600008+0000","last_active":"2023-12-14T18:24:41.941747+0000","last_peered":"2023-12-14T18:24:41.941747+0000","last_clean":"2023-12-14T18:24:41.941747+0000","last_became_active":"2023-12-14T18:24:40.599596+0000","last_became_peered":"2023-12-14T18:24:40.599596+0000","last_unstale":"2023-12-14T18:24:41.941747+0000","last_undegraded":"2023-12-14T18:24:41.941747+0000","last_fullsized":"2023-12-14T18:24:41.941747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:10:10.770024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095629+0000","last_change":"2023-12-14T18:24:36.176569+0000","last_active":"2023-12-14T18:24:41.095629+0000","last_peered":"2023-12-14T18:24:41.095629+0000","last_clean":"2023-12-14T18:24:41.095629+0000","last_became_active":"2023-12-14T18:24:36.176043+0000","last_became_peered":"2023-12-14T18:24:36.176043+0000","last_unstale":"2023-12-14T18:24:41.095629+0000","last_undegraded":"2023-12-14T18:24:41.095629+0000","last_fullsized":"2023-12-14T18:24:41.095629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:10:44.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847627+0000","last_change":"2023-12-14T18:24:36.593761+0000","last_active":"2023-12-14T18:25:36.847627+0000","last_peered":"2023-12-14T18:25:36.847627+0000","last_clean":"2023-12-14T18:25:36.847627+0000","last_became_active":"2023-12-14T18:24:36.593452+0000","last_became_peered":"2023-12-14T18:24:36.593452+0000","last_unstale":"2023-12-14T18:25:36.847627+0000","last_undegraded":"2023-12-14T18:25:36.847627+0000","last_fullsized":"2023-12-14T18:25:36.847627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:36:25.313521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001069+0000","last_change":"2023-12-14T18:24:40.598064+0000","last_active":"2023-12-14T18:24:41.001069+0000","last_peered":"2023-12-14T18:24:41.001069+0000","last_clean":"2023-12-14T18:24:41.001069+0000","last_became_active":"2023-12-14T18:24:40.597672+0000","last_became_peered":"2023-12-14T18:24:40.597672+0000","last_unstale":"2023-12-14T18:24:41.001069+0000","last_undegraded":"2023-12-14T18:24:41.001069+0000","last_fullsized":"2023-12-14T18:24:41.001069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:41:54.138227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":69,"reported_epoch":84,"state":"peering","last_fresh":"2023-12-14T18:25:36.813236+0000","last_change":"2023-12-14T18:25:36.845980+0000","last_active":"2023-12-14T18:25:36.845980+0000","last_peered":"2023-12-14T18:25:36.813236+0000","last_clean":"2023-12-14T18:25:36.813236+0000","last_became_active":"2023-12-14T18:24:36.175965+0000","last_became_peered":"2023-12-14T18:24:36.175965+0000","last_unstale":"2023-12-14T18:25:36.813236+0000","last_undegraded":"2023-12-14T18:25:36.813236+0000","last_fullsized":"2023-12-14T18:25:36.813236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095492+0000","last_change":"2023-12-14T18:24:36.583687+0000","last_active":"2023-12-14T18:24:41.095492+0000","last_peered":"2023-12-14T18:24:41.095492+0000","last_clean":"2023-12-14T18:24:41.095492+0000","last_became_active":"2023-12-14T18:24:36.583047+0000","last_became_peered":"2023-12-14T18:24:36.583047+0000","last_unstale":"2023-12-14T18:24:41.095492+0000","last_undegraded":"2023-12-14T18:24:41.095492+0000","last_fullsized":"2023-12-14T18:24:41.095492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:26:10.095059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001102+0000","last_change":"2023-12-14T18:24:40.605217+0000","last_active":"2023-12-14T18:24:41.001102+0000","last_peered":"2023-12-14T18:24:41.001102+0000","last_clean":"2023-12-14T18:24:41.001102+0000","last_became_active":"2023-12-14T18:24:40.604807+0000","last_became_peered":"2023-12-14T18:24:40.604807+0000","last_unstale":"2023-12-14T18:24:41.001102+0000","last_undegraded":"2023-12-14T18:24:41.001102+0000","last_fullsized":"2023-12-14T18:24:41.001102+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:48:27.699822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767627+0000","last_change":"2023-12-14T18:24:36.174875+0000","last_active":"2023-12-14T18:25:00.767627+0000","last_peered":"2023-12-14T18:25:00.767627+0000","last_clean":"2023-12-14T18:25:00.767627+0000","last_became_active":"2023-12-14T18:24:36.173984+0000","last_became_peered":"2023-12-14T18:24:36.173984+0000","last_unstale":"2023-12-14T18:25:00.767627+0000","last_undegraded":"2023-12-14T18:25:00.767627+0000","last_fullsized":"2023-12-14T18:25:00.767627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:24:27.753878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847649+0000","last_change":"2023-12-14T18:24:36.580563+0000","last_active":"2023-12-14T18:25:36.847649+0000","last_peered":"2023-12-14T18:25:36.847649+0000","last_clean":"2023-12-14T18:25:36.847649+0000","last_became_active":"2023-12-14T18:24:36.580196+0000","last_became_peered":"2023-12-14T18:24:36.580196+0000","last_unstale":"2023-12-14T18:25:36.847649+0000","last_undegraded":"2023-12-14T18:25:36.847649+0000","last_fullsized":"2023-12-14T18:25:36.847649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:22:54.569465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939735+0000","last_change":"2023-12-14T18:24:40.601223+0000","last_active":"2023-12-14T18:24:41.939735+0000","last_peered":"2023-12-14T18:24:41.939735+0000","last_clean":"2023-12-14T18:24:41.939735+0000","last_became_active":"2023-12-14T18:24:40.600795+0000","last_became_peered":"2023-12-14T18:24:40.600795+0000","last_unstale":"2023-12-14T18:24:41.939735+0000","last_undegraded":"2023-12-14T18:24:41.939735+0000","last_fullsized":"2023-12-14T18:24:41.939735+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:27:49.251012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767406+0000","last_change":"2023-12-14T18:24:36.173600+0000","last_active":"2023-12-14T18:25:00.767406+0000","last_peered":"2023-12-14T18:25:00.767406+0000","last_clean":"2023-12-14T18:25:00.767406+0000","last_became_active":"2023-12-14T18:24:36.173272+0000","last_became_peered":"2023-12-14T18:24:36.173272+0000","last_unstale":"2023-12-14T18:25:00.767406+0000","last_undegraded":"2023-12-14T18:25:00.767406+0000","last_fullsized":"2023-12-14T18:25:00.767406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:23.589714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998814+0000","last_change":"2023-12-14T18:24:36.575060+0000","last_active":"2023-12-14T18:24:40.998814+0000","last_peered":"2023-12-14T18:24:40.998814+0000","last_clean":"2023-12-14T18:24:40.998814+0000","last_became_active":"2023-12-14T18:24:36.574749+0000","last_became_peered":"2023-12-14T18:24:36.574749+0000","last_unstale":"2023-12-14T18:24:40.998814+0000","last_undegraded":"2023-12-14T18:24:40.998814+0000","last_fullsized":"2023-12-14T18:24:40.998814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:44:44.163762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939489+0000","last_change":"2023-12-14T18:24:40.605431+0000","last_active":"2023-12-14T18:24:41.939489+0000","last_peered":"2023-12-14T18:24:41.939489+0000","last_clean":"2023-12-14T18:24:41.939489+0000","last_became_active":"2023-12-14T18:24:40.604951+0000","last_became_peered":"2023-12-14T18:24:40.604951+0000","last_unstale":"2023-12-14T18:24:41.939489+0000","last_undegraded":"2023-12-14T18:24:41.939489+0000","last_fullsized":"2023-12-14T18:24:41.939489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:46:36.552203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.846899+0000","last_change":"2023-12-14T18:24:36.175493+0000","last_active":"2023-12-14T18:25:36.846899+0000","last_peered":"2023-12-14T18:25:36.846899+0000","last_clean":"2023-12-14T18:25:36.846899+0000","last_became_active":"2023-12-14T18:24:36.175205+0000","last_became_peered":"2023-12-14T18:24:36.175205+0000","last_unstale":"2023-12-14T18:25:36.846899+0000","last_undegraded":"2023-12-14T18:25:36.846899+0000","last_fullsized":"2023-12-14T18:25:36.846899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:06:22.692820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092822+0000","last_change":"2023-12-14T18:24:36.587218+0000","last_active":"2023-12-14T18:24:41.092822+0000","last_peered":"2023-12-14T18:24:41.092822+0000","last_clean":"2023-12-14T18:24:41.092822+0000","last_became_active":"2023-12-14T18:24:36.586910+0000","last_became_peered":"2023-12-14T18:24:36.586910+0000","last_unstale":"2023-12-14T18:24:41.092822+0000","last_undegraded":"2023-12-14T18:24:41.092822+0000","last_fullsized":"2023-12-14T18:24:41.092822+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:04.691462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999151+0000","last_change":"2023-12-14T18:24:40.598409+0000","last_active":"2023-12-14T18:24:40.999151+0000","last_peered":"2023-12-14T18:24:40.999151+0000","last_clean":"2023-12-14T18:24:40.999151+0000","last_became_active":"2023-12-14T18:24:40.598011+0000","last_became_peered":"2023-12-14T18:24:40.598011+0000","last_unstale":"2023-12-14T18:24:40.999151+0000","last_undegraded":"2023-12-14T18:24:40.999151+0000","last_fullsized":"2023-12-14T18:24:40.999151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:30.946044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:34.810413+0000","last_change":"2023-12-14T18:24:36.172277+0000","last_active":"2023-12-14T18:25:34.810413+0000","last_peered":"2023-12-14T18:25:34.810413+0000","last_clean":"2023-12-14T18:25:34.810413+0000","last_became_active":"2023-12-14T18:24:36.171907+0000","last_became_peered":"2023-12-14T18:24:36.171907+0000","last_unstale":"2023-12-14T18:25:34.810413+0000","last_undegraded":"2023-12-14T18:25:34.810413+0000","last_fullsized":"2023-12-14T18:25:34.810413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:45:12.002124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.844992+0000","last_change":"2023-12-14T18:24:36.584783+0000","last_active":"2023-12-14T18:25:36.844992+0000","last_peered":"2023-12-14T18:25:36.844992+0000","last_clean":"2023-12-14T18:25:36.844992+0000","last_became_active":"2023-12-14T18:24:36.584424+0000","last_became_peered":"2023-12-14T18:24:36.584424+0000","last_unstale":"2023-12-14T18:25:36.844992+0000","last_undegraded":"2023-12-14T18:25:36.844992+0000","last_fullsized":"2023-12-14T18:25:36.844992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:54:02.928872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092851+0000","last_change":"2023-12-14T18:24:40.603176+0000","last_active":"2023-12-14T18:24:41.092851+0000","last_peered":"2023-12-14T18:24:41.092851+0000","last_clean":"2023-12-14T18:24:41.092851+0000","last_became_active":"2023-12-14T18:24:40.602868+0000","last_became_peered":"2023-12-14T18:24:40.602868+0000","last_unstale":"2023-12-14T18:24:41.092851+0000","last_undegraded":"2023-12-14T18:24:41.092851+0000","last_fullsized":"2023-12-14T18:24:41.092851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:02:23.455860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844517+0000","last_change":"2023-12-14T18:24:36.177638+0000","last_active":"2023-12-14T18:25:37.844517+0000","last_peered":"2023-12-14T18:25:37.844517+0000","last_clean":"2023-12-14T18:25:37.844517+0000","last_became_active":"2023-12-14T18:24:36.177335+0000","last_became_peered":"2023-12-14T18:24:36.177335+0000","last_unstale":"2023-12-14T18:25:37.844517+0000","last_undegraded":"2023-12-14T18:25:37.844517+0000","last_fullsized":"2023-12-14T18:25:37.844517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:29:24.204595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939828+0000","last_change":"2023-12-14T18:24:36.577508+0000","last_active":"2023-12-14T18:24:41.939828+0000","last_peered":"2023-12-14T18:24:41.939828+0000","last_clean":"2023-12-14T18:24:41.939828+0000","last_became_active":"2023-12-14T18:24:36.577196+0000","last_became_peered":"2023-12-14T18:24:36.577196+0000","last_unstale":"2023-12-14T18:24:41.939828+0000","last_undegraded":"2023-12-14T18:24:41.939828+0000","last_fullsized":"2023-12-14T18:24:41.939828+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:32:16.278415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941240+0000","last_change":"2023-12-14T18:24:40.594816+0000","last_active":"2023-12-14T18:24:41.941240+0000","last_peered":"2023-12-14T18:24:41.941240+0000","last_clean":"2023-12-14T18:24:41.941240+0000","last_became_active":"2023-12-14T18:24:40.594371+0000","last_became_peered":"2023-12-14T18:24:40.594371+0000","last_unstale":"2023-12-14T18:24:41.941240+0000","last_undegraded":"2023-12-14T18:24:41.941240+0000","last_fullsized":"2023-12-14T18:24:41.941240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:40:46.343094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847519+0000","last_change":"2023-12-14T18:24:36.174238+0000","last_active":"2023-12-14T18:25:36.847519+0000","last_peered":"2023-12-14T18:25:36.847519+0000","last_clean":"2023-12-14T18:25:36.847519+0000","last_became_active":"2023-12-14T18:24:36.173373+0000","last_became_peered":"2023-12-14T18:24:36.173373+0000","last_unstale":"2023-12-14T18:25:36.847519+0000","last_undegraded":"2023-12-14T18:25:36.847519+0000","last_fullsized":"2023-12-14T18:25:36.847519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:24:20.899377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.844664+0000","last_change":"2023-12-14T18:24:36.588589+0000","last_active":"2023-12-14T18:25:36.844664+0000","last_peered":"2023-12-14T18:25:36.844664+0000","last_clean":"2023-12-14T18:25:36.844664+0000","last_became_active":"2023-12-14T18:24:36.588269+0000","last_became_peered":"2023-12-14T18:24:36.588269+0000","last_unstale":"2023-12-14T18:25:36.844664+0000","last_undegraded":"2023-12-14T18:25:36.844664+0000","last_fullsized":"2023-12-14T18:25:36.844664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:23.993313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.092841+0000","last_change":"2023-12-14T18:24:40.601333+0000","last_active":"2023-12-14T18:24:41.092841+0000","last_peered":"2023-12-14T18:24:41.092841+0000","last_clean":"2023-12-14T18:24:41.092841+0000","last_became_active":"2023-12-14T18:24:40.600966+0000","last_became_peered":"2023-12-14T18:24:40.600966+0000","last_unstale":"2023-12-14T18:24:41.092841+0000","last_undegraded":"2023-12-14T18:24:41.092841+0000","last_fullsized":"2023-12-14T18:24:41.092841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:55:36.433276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":133,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:35.811430+0000","last_change":"2023-12-14T18:24:36.177299+0000","last_active":"2023-12-14T18:25:35.811430+0000","last_peered":"2023-12-14T18:25:35.811430+0000","last_clean":"2023-12-14T18:25:35.811430+0000","last_became_active":"2023-12-14T18:24:36.176971+0000","last_became_peered":"2023-12-14T18:24:36.176971+0000","last_unstale":"2023-12-14T18:25:35.811430+0000","last_undegraded":"2023-12-14T18:25:35.811430+0000","last_fullsized":"2023-12-14T18:25:35.811430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:02:46.083153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60836,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":80,"num_read_kb":99,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095663+0000","last_change":"2023-12-14T18:24:36.586315+0000","last_active":"2023-12-14T18:24:41.095663+0000","last_peered":"2023-12-14T18:24:41.095663+0000","last_clean":"2023-12-14T18:24:41.095663+0000","last_became_active":"2023-12-14T18:24:36.586000+0000","last_became_peered":"2023-12-14T18:24:36.586000+0000","last_unstale":"2023-12-14T18:24:41.095663+0000","last_undegraded":"2023-12-14T18:24:41.095663+0000","last_fullsized":"2023-12-14T18:24:41.095663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:19:55.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847904+0000","last_change":"2023-12-14T18:24:40.596973+0000","last_active":"2023-12-14T18:25:36.847904+0000","last_peered":"2023-12-14T18:25:36.847904+0000","last_clean":"2023-12-14T18:25:36.847904+0000","last_became_active":"2023-12-14T18:24:40.596355+0000","last_became_peered":"2023-12-14T18:24:40.596355+0000","last_unstale":"2023-12-14T18:25:36.847904+0000","last_undegraded":"2023-12-14T18:25:36.847904+0000","last_fullsized":"2023-12-14T18:25:36.847904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:42:06.172578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":29,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767785+0000","last_change":"2023-12-14T18:24:36.175287+0000","last_active":"2023-12-14T18:25:00.767785+0000","last_peered":"2023-12-14T18:25:00.767785+0000","last_clean":"2023-12-14T18:25:00.767785+0000","last_became_active":"2023-12-14T18:24:36.174952+0000","last_became_peered":"2023-12-14T18:24:36.174952+0000","last_unstale":"2023-12-14T18:25:00.767785+0000","last_undegraded":"2023-12-14T18:25:00.767785+0000","last_fullsized":"2023-12-14T18:25:00.767785+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:44:43.661729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940588+0000","last_change":"2023-12-14T18:24:36.582314+0000","last_active":"2023-12-14T18:24:41.940588+0000","last_peered":"2023-12-14T18:24:41.940588+0000","last_clean":"2023-12-14T18:24:41.940588+0000","last_became_active":"2023-12-14T18:24:36.581963+0000","last_became_peered":"2023-12-14T18:24:36.581963+0000","last_unstale":"2023-12-14T18:24:41.940588+0000","last_undegraded":"2023-12-14T18:24:41.940588+0000","last_fullsized":"2023-12-14T18:24:41.940588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:02:38.477328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939741+0000","last_change":"2023-12-14T18:24:40.599979+0000","last_active":"2023-12-14T18:24:41.939741+0000","last_peered":"2023-12-14T18:24:41.939741+0000","last_clean":"2023-12-14T18:24:41.939741+0000","last_became_active":"2023-12-14T18:24:40.599258+0000","last_became_peered":"2023-12-14T18:24:40.599258+0000","last_unstale":"2023-12-14T18:24:41.939741+0000","last_undegraded":"2023-12-14T18:24:41.939741+0000","last_fullsized":"2023-12-14T18:24:41.939741+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:06.783797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847541+0000","last_change":"2023-12-14T18:24:36.175768+0000","last_active":"2023-12-14T18:25:36.847541+0000","last_peered":"2023-12-14T18:25:36.847541+0000","last_clean":"2023-12-14T18:25:36.847541+0000","last_became_active":"2023-12-14T18:24:36.174973+0000","last_became_peered":"2023-12-14T18:24:36.174973+0000","last_unstale":"2023-12-14T18:25:36.847541+0000","last_undegraded":"2023-12-14T18:25:36.847541+0000","last_fullsized":"2023-12-14T18:25:36.847541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:55:13.193815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844716+0000","last_change":"2023-12-14T18:24:36.591856+0000","last_active":"2023-12-14T18:25:37.844716+0000","last_peered":"2023-12-14T18:25:37.844716+0000","last_clean":"2023-12-14T18:25:37.844716+0000","last_became_active":"2023-12-14T18:24:36.591554+0000","last_became_peered":"2023-12-14T18:24:36.591554+0000","last_unstale":"2023-12-14T18:25:37.844716+0000","last_undegraded":"2023-12-14T18:25:37.844716+0000","last_fullsized":"2023-12-14T18:25:37.844716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:07:38.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093250+0000","last_change":"2023-12-14T18:24:40.603794+0000","last_active":"2023-12-14T18:24:41.093250+0000","last_peered":"2023-12-14T18:24:41.093250+0000","last_clean":"2023-12-14T18:24:41.093250+0000","last_became_active":"2023-12-14T18:24:40.603311+0000","last_became_peered":"2023-12-14T18:24:40.603311+0000","last_unstale":"2023-12-14T18:24:41.093250+0000","last_undegraded":"2023-12-14T18:24:41.093250+0000","last_fullsized":"2023-12-14T18:24:41.093250+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:11:14.354480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:33.832001+0000","last_change":"2023-12-14T18:24:36.177411+0000","last_active":"2023-12-14T18:25:33.832001+0000","last_peered":"2023-12-14T18:25:33.832001+0000","last_clean":"2023-12-14T18:25:33.832001+0000","last_became_active":"2023-12-14T18:24:36.177047+0000","last_became_peered":"2023-12-14T18:24:36.177047+0000","last_unstale":"2023-12-14T18:25:33.832001+0000","last_undegraded":"2023-12-14T18:25:33.832001+0000","last_fullsized":"2023-12-14T18:25:33.832001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:58:22.871429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"activating","last_fresh":"2023-12-14T18:25:37.844035+0000","last_change":"2023-12-14T18:25:37.844035+0000","last_active":"2023-12-14T18:25:36.841095+0000","last_peered":"2023-12-14T18:24:41.939941+0000","last_clean":"2023-12-14T18:24:41.939941+0000","last_became_active":"2023-12-14T18:24:36.596900+0000","last_became_peered":"2023-12-14T18:24:36.596900+0000","last_unstale":"2023-12-14T18:25:37.844035+0000","last_undegraded":"2023-12-14T18:25:37.844035+0000","last_fullsized":"2023-12-14T18:25:37.844035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:19:00.666432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":["7","5","9","10"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940190+0000","last_change":"2023-12-14T18:24:40.607042+0000","last_active":"2023-12-14T18:24:41.940190+0000","last_peered":"2023-12-14T18:24:41.940190+0000","last_clean":"2023-12-14T18:24:41.940190+0000","last_became_active":"2023-12-14T18:24:40.606563+0000","last_became_peered":"2023-12-14T18:24:40.606563+0000","last_unstale":"2023-12-14T18:24:41.940190+0000","last_undegraded":"2023-12-14T18:24:41.940190+0000","last_fullsized":"2023-12-14T18:24:41.940190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:31:29.839031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.769004+0000","last_change":"2023-12-14T18:24:36.171994+0000","last_active":"2023-12-14T18:25:00.769004+0000","last_peered":"2023-12-14T18:25:00.769004+0000","last_clean":"2023-12-14T18:25:00.769004+0000","last_became_active":"2023-12-14T18:24:36.171661+0000","last_became_peered":"2023-12-14T18:24:36.171661+0000","last_unstale":"2023-12-14T18:25:00.769004+0000","last_undegraded":"2023-12-14T18:25:00.769004+0000","last_fullsized":"2023-12-14T18:25:00.769004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:33:33.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847106+0000","last_change":"2023-12-14T18:24:36.577818+0000","last_active":"2023-12-14T18:25:36.847106+0000","last_peered":"2023-12-14T18:25:36.847106+0000","last_clean":"2023-12-14T18:25:36.847106+0000","last_became_active":"2023-12-14T18:24:36.577356+0000","last_became_peered":"2023-12-14T18:24:36.577356+0000","last_unstale":"2023-12-14T18:25:36.847106+0000","last_undegraded":"2023-12-14T18:25:36.847106+0000","last_fullsized":"2023-12-14T18:25:36.847106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:19:22.337928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847031+0000","last_change":"2023-12-14T18:24:40.601995+0000","last_active":"2023-12-14T18:25:36.847031+0000","last_peered":"2023-12-14T18:25:36.847031+0000","last_clean":"2023-12-14T18:25:36.847031+0000","last_became_active":"2023-12-14T18:24:40.601571+0000","last_became_peered":"2023-12-14T18:24:40.601571+0000","last_unstale":"2023-12-14T18:25:36.847031+0000","last_undegraded":"2023-12-14T18:25:36.847031+0000","last_fullsized":"2023-12-14T18:25:36.847031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:40:59.842261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.756794+0000","last_change":"2023-12-14T18:24:36.168661+0000","last_active":"2023-12-14T18:25:00.756794+0000","last_peered":"2023-12-14T18:25:00.756794+0000","last_clean":"2023-12-14T18:25:00.756794+0000","last_became_active":"2023-12-14T18:24:36.168299+0000","last_became_peered":"2023-12-14T18:24:36.168299+0000","last_unstale":"2023-12-14T18:25:00.756794+0000","last_undegraded":"2023-12-14T18:25:00.756794+0000","last_fullsized":"2023-12-14T18:25:00.756794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:55.288902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939900+0000","last_change":"2023-12-14T18:24:36.578417+0000","last_active":"2023-12-14T18:24:41.939900+0000","last_peered":"2023-12-14T18:24:41.939900+0000","last_clean":"2023-12-14T18:24:41.939900+0000","last_became_active":"2023-12-14T18:24:36.578080+0000","last_became_peered":"2023-12-14T18:24:36.578080+0000","last_unstale":"2023-12-14T18:24:41.939900+0000","last_undegraded":"2023-12-14T18:24:41.939900+0000","last_fullsized":"2023-12-14T18:24:41.939900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:07:18.978037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095510+0000","last_change":"2023-12-14T18:24:40.771866+0000","last_active":"2023-12-14T18:24:41.095510+0000","last_peered":"2023-12-14T18:24:41.095510+0000","last_clean":"2023-12-14T18:24:41.095510+0000","last_became_active":"2023-12-14T18:24:40.771324+0000","last_became_peered":"2023-12-14T18:24:40.771324+0000","last_unstale":"2023-12-14T18:24:41.095510+0000","last_undegraded":"2023-12-14T18:24:41.095510+0000","last_fullsized":"2023-12-14T18:24:41.095510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:20:19.058538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":22,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093047+0000","last_change":"2023-12-14T18:24:36.170489+0000","last_active":"2023-12-14T18:24:41.093047+0000","last_peered":"2023-12-14T18:24:41.093047+0000","last_clean":"2023-12-14T18:24:41.093047+0000","last_became_active":"2023-12-14T18:24:36.170161+0000","last_became_peered":"2023-12-14T18:24:36.170161+0000","last_unstale":"2023-12-14T18:24:41.093047+0000","last_undegraded":"2023-12-14T18:24:41.093047+0000","last_fullsized":"2023-12-14T18:24:41.093047+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:33:56.690387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940761+0000","last_change":"2023-12-14T18:24:36.591094+0000","last_active":"2023-12-14T18:24:41.940761+0000","last_peered":"2023-12-14T18:24:41.940761+0000","last_clean":"2023-12-14T18:24:41.940761+0000","last_became_active":"2023-12-14T18:24:36.590779+0000","last_became_peered":"2023-12-14T18:24:36.590779+0000","last_unstale":"2023-12-14T18:24:41.940761+0000","last_undegraded":"2023-12-14T18:24:41.940761+0000","last_fullsized":"2023-12-14T18:24:41.940761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:16:00.885373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940628+0000","last_change":"2023-12-14T18:24:40.604125+0000","last_active":"2023-12-14T18:24:41.940628+0000","last_peered":"2023-12-14T18:24:41.940628+0000","last_clean":"2023-12-14T18:24:41.940628+0000","last_became_active":"2023-12-14T18:24:40.603242+0000","last_became_peered":"2023-12-14T18:24:40.603242+0000","last_unstale":"2023-12-14T18:24:41.940628+0000","last_undegraded":"2023-12-14T18:24:41.940628+0000","last_fullsized":"2023-12-14T18:24:41.940628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:52:35.630493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845182+0000","last_change":"2023-12-14T18:24:36.177567+0000","last_active":"2023-12-14T18:25:37.845182+0000","last_peered":"2023-12-14T18:25:37.845182+0000","last_clean":"2023-12-14T18:25:37.845182+0000","last_became_active":"2023-12-14T18:24:36.176949+0000","last_became_peered":"2023-12-14T18:24:36.176949+0000","last_unstale":"2023-12-14T18:25:37.845182+0000","last_undegraded":"2023-12-14T18:25:37.845182+0000","last_fullsized":"2023-12-14T18:25:37.845182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:41:12.145991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940032+0000","last_change":"2023-12-14T18:24:36.594224+0000","last_active":"2023-12-14T18:24:41.940032+0000","last_peered":"2023-12-14T18:24:41.940032+0000","last_clean":"2023-12-14T18:24:41.940032+0000","last_became_active":"2023-12-14T18:24:36.593859+0000","last_became_peered":"2023-12-14T18:24:36.593859+0000","last_unstale":"2023-12-14T18:24:41.940032+0000","last_undegraded":"2023-12-14T18:24:41.940032+0000","last_fullsized":"2023-12-14T18:24:41.940032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:59:04.044943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940173+0000","last_change":"2023-12-14T18:24:40.771814+0000","last_active":"2023-12-14T18:24:41.940173+0000","last_peered":"2023-12-14T18:24:41.940173+0000","last_clean":"2023-12-14T18:24:41.940173+0000","last_became_active":"2023-12-14T18:24:40.771387+0000","last_became_peered":"2023-12-14T18:24:40.771387+0000","last_unstale":"2023-12-14T18:24:41.940173+0000","last_undegraded":"2023-12-14T18:24:41.940173+0000","last_fullsized":"2023-12-14T18:24:41.940173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:22.744059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.846862+0000","last_change":"2023-12-14T18:24:36.588826+0000","last_active":"2023-12-14T18:25:36.846862+0000","last_peered":"2023-12-14T18:25:36.846862+0000","last_clean":"2023-12-14T18:25:36.846862+0000","last_became_active":"2023-12-14T18:24:36.588320+0000","last_became_peered":"2023-12-14T18:24:36.588320+0000","last_unstale":"2023-12-14T18:25:36.846862+0000","last_undegraded":"2023-12-14T18:25:36.846862+0000","last_fullsized":"2023-12-14T18:25:36.846862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:52:29.540309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:34.832934+0000","last_change":"2023-12-14T18:24:36.173995+0000","last_active":"2023-12-14T18:25:34.832934+0000","last_peered":"2023-12-14T18:25:34.832934+0000","last_clean":"2023-12-14T18:25:34.832934+0000","last_became_active":"2023-12-14T18:24:36.173381+0000","last_became_peered":"2023-12-14T18:24:36.173381+0000","last_unstale":"2023-12-14T18:25:34.832934+0000","last_undegraded":"2023-12-14T18:25:34.832934+0000","last_fullsized":"2023-12-14T18:25:34.832934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:17:33.452268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844039+0000","last_change":"2023-12-14T18:24:40.605099+0000","last_active":"2023-12-14T18:25:37.844039+0000","last_peered":"2023-12-14T18:25:37.844039+0000","last_clean":"2023-12-14T18:25:37.844039+0000","last_became_active":"2023-12-14T18:24:40.604773+0000","last_became_peered":"2023-12-14T18:24:40.604773+0000","last_unstale":"2023-12-14T18:25:37.844039+0000","last_undegraded":"2023-12-14T18:25:37.844039+0000","last_fullsized":"2023-12-14T18:25:37.844039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:03.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998989+0000","last_change":"2023-12-14T18:24:36.574993+0000","last_active":"2023-12-14T18:24:40.998989+0000","last_peered":"2023-12-14T18:24:40.998989+0000","last_clean":"2023-12-14T18:24:40.998989+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:24:40.998989+0000","last_undegraded":"2023-12-14T18:24:40.998989+0000","last_fullsized":"2023-12-14T18:24:40.998989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:36:49.384865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767286+0000","last_change":"2023-12-14T18:24:36.170061+0000","last_active":"2023-12-14T18:25:00.767286+0000","last_peered":"2023-12-14T18:25:00.767286+0000","last_clean":"2023-12-14T18:25:00.767286+0000","last_became_active":"2023-12-14T18:24:36.169652+0000","last_became_peered":"2023-12-14T18:24:36.169652+0000","last_unstale":"2023-12-14T18:25:00.767286+0000","last_undegraded":"2023-12-14T18:25:00.767286+0000","last_fullsized":"2023-12-14T18:25:00.767286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:35:33.204173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843576+0000","last_change":"2023-12-14T18:23:49.076644+0000","last_active":"2023-12-14T18:25:37.843576+0000","last_peered":"2023-12-14T18:25:37.843576+0000","last_clean":"2023-12-14T18:25:37.843576+0000","last_became_active":"2023-12-14T18:23:49.076263+0000","last_became_peered":"2023-12-14T18:23:49.076263+0000","last_unstale":"2023-12-14T18:25:37.843576+0000","last_undegraded":"2023-12-14T18:25:37.843576+0000","last_fullsized":"2023-12-14T18:25:37.843576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_clean_scrub_stamp":"2023-12-14T18:21:24.715131+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:26:53.561034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847479+0000","last_change":"2023-12-14T18:24:40.772015+0000","last_active":"2023-12-14T18:25:36.847479+0000","last_peered":"2023-12-14T18:25:36.847479+0000","last_clean":"2023-12-14T18:25:36.847479+0000","last_became_active":"2023-12-14T18:24:40.771497+0000","last_became_peered":"2023-12-14T18:24:40.771497+0000","last_unstale":"2023-12-14T18:25:36.847479+0000","last_undegraded":"2023-12-14T18:25:36.847479+0000","last_fullsized":"2023-12-14T18:25:36.847479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:07:56.111457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939552+0000","last_change":"2023-12-14T18:24:36.584402+0000","last_active":"2023-12-14T18:24:41.939552+0000","last_peered":"2023-12-14T18:24:41.939552+0000","last_clean":"2023-12-14T18:24:41.939552+0000","last_became_active":"2023-12-14T18:24:36.584177+0000","last_became_peered":"2023-12-14T18:24:36.584177+0000","last_unstale":"2023-12-14T18:24:41.939552+0000","last_undegraded":"2023-12-14T18:24:41.939552+0000","last_fullsized":"2023-12-14T18:24:41.939552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:38:45.449137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843698+0000","last_change":"2023-12-14T18:24:36.175017+0000","last_active":"2023-12-14T18:25:37.843698+0000","last_peered":"2023-12-14T18:25:37.843698+0000","last_clean":"2023-12-14T18:25:37.843698+0000","last_became_active":"2023-12-14T18:24:36.174639+0000","last_became_peered":"2023-12-14T18:24:36.174639+0000","last_unstale":"2023-12-14T18:25:37.843698+0000","last_undegraded":"2023-12-14T18:25:37.843698+0000","last_fullsized":"2023-12-14T18:25:37.843698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:31:26.105116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939585+0000","last_change":"2023-12-14T18:24:40.596043+0000","last_active":"2023-12-14T18:24:41.939585+0000","last_peered":"2023-12-14T18:24:41.939585+0000","last_clean":"2023-12-14T18:24:41.939585+0000","last_became_active":"2023-12-14T18:24:40.595603+0000","last_became_peered":"2023-12-14T18:24:40.595603+0000","last_unstale":"2023-12-14T18:24:41.939585+0000","last_undegraded":"2023-12-14T18:24:41.939585+0000","last_fullsized":"2023-12-14T18:24:41.939585+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:09:47.437024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.038437+0000","last_change":"2023-12-14T18:24:36.177484+0000","last_active":"2023-12-14T18:25:00.038437+0000","last_peered":"2023-12-14T18:25:00.038437+0000","last_clean":"2023-12-14T18:25:00.038437+0000","last_became_active":"2023-12-14T18:24:36.176592+0000","last_became_peered":"2023-12-14T18:24:36.176592+0000","last_unstale":"2023-12-14T18:25:00.038437+0000","last_undegraded":"2023-12-14T18:25:00.038437+0000","last_fullsized":"2023-12-14T18:25:00.038437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:21:55.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940747+0000","last_change":"2023-12-14T18:24:36.585687+0000","last_active":"2023-12-14T18:24:41.940747+0000","last_peered":"2023-12-14T18:24:41.940747+0000","last_clean":"2023-12-14T18:24:41.940747+0000","last_became_active":"2023-12-14T18:24:36.585360+0000","last_became_peered":"2023-12-14T18:24:36.585360+0000","last_unstale":"2023-12-14T18:24:41.940747+0000","last_undegraded":"2023-12-14T18:24:41.940747+0000","last_fullsized":"2023-12-14T18:24:41.940747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:42.737826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998830+0000","last_change":"2023-12-14T18:24:40.771747+0000","last_active":"2023-12-14T18:24:40.998830+0000","last_peered":"2023-12-14T18:24:40.998830+0000","last_clean":"2023-12-14T18:24:40.998830+0000","last_became_active":"2023-12-14T18:24:40.771310+0000","last_became_peered":"2023-12-14T18:24:40.771310+0000","last_unstale":"2023-12-14T18:24:40.998830+0000","last_undegraded":"2023-12-14T18:24:40.998830+0000","last_fullsized":"2023-12-14T18:24:40.998830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:52.843438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:06.093093+0000","last_change":"2023-12-14T18:24:36.171833+0000","last_active":"2023-12-14T18:25:06.093093+0000","last_peered":"2023-12-14T18:25:06.093093+0000","last_clean":"2023-12-14T18:25:06.093093+0000","last_became_active":"2023-12-14T18:24:36.171512+0000","last_became_peered":"2023-12-14T18:24:36.171512+0000","last_unstale":"2023-12-14T18:25:06.093093+0000","last_undegraded":"2023-12-14T18:25:06.093093+0000","last_fullsized":"2023-12-14T18:25:06.093093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:56.553040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999452+0000","last_change":"2023-12-14T18:24:36.594938+0000","last_active":"2023-12-14T18:24:40.999452+0000","last_peered":"2023-12-14T18:24:40.999452+0000","last_clean":"2023-12-14T18:24:40.999452+0000","last_became_active":"2023-12-14T18:24:36.594573+0000","last_became_peered":"2023-12-14T18:24:36.594573+0000","last_unstale":"2023-12-14T18:24:40.999452+0000","last_undegraded":"2023-12-14T18:24:40.999452+0000","last_fullsized":"2023-12-14T18:24:40.999452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:18:30.843187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000512+0000","last_change":"2023-12-14T18:24:40.601146+0000","last_active":"2023-12-14T18:24:41.000512+0000","last_peered":"2023-12-14T18:24:41.000512+0000","last_clean":"2023-12-14T18:24:41.000512+0000","last_became_active":"2023-12-14T18:24:40.600745+0000","last_became_peered":"2023-12-14T18:24:40.600745+0000","last_unstale":"2023-12-14T18:24:41.000512+0000","last_undegraded":"2023-12-14T18:24:41.000512+0000","last_fullsized":"2023-12-14T18:24:41.000512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:48:20.003480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039339+0000","last_change":"2023-12-14T18:24:36.175880+0000","last_active":"2023-12-14T18:25:00.039339+0000","last_peered":"2023-12-14T18:25:00.039339+0000","last_clean":"2023-12-14T18:25:00.039339+0000","last_became_active":"2023-12-14T18:24:36.175514+0000","last_became_peered":"2023-12-14T18:24:36.175514+0000","last_unstale":"2023-12-14T18:25:00.039339+0000","last_undegraded":"2023-12-14T18:25:00.039339+0000","last_fullsized":"2023-12-14T18:25:00.039339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:00:40.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939608+0000","last_change":"2023-12-14T18:24:36.577062+0000","last_active":"2023-12-14T18:24:41.939608+0000","last_peered":"2023-12-14T18:24:41.939608+0000","last_clean":"2023-12-14T18:24:41.939608+0000","last_became_active":"2023-12-14T18:24:36.576733+0000","last_became_peered":"2023-12-14T18:24:36.576733+0000","last_unstale":"2023-12-14T18:24:41.939608+0000","last_undegraded":"2023-12-14T18:24:41.939608+0000","last_fullsized":"2023-12-14T18:24:41.939608+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:12:08.207474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940397+0000","last_change":"2023-12-14T18:24:40.772752+0000","last_active":"2023-12-14T18:24:41.940397+0000","last_peered":"2023-12-14T18:24:41.940397+0000","last_clean":"2023-12-14T18:24:41.940397+0000","last_became_active":"2023-12-14T18:24:40.772168+0000","last_became_peered":"2023-12-14T18:24:40.772168+0000","last_unstale":"2023-12-14T18:24:41.940397+0000","last_undegraded":"2023-12-14T18:24:41.940397+0000","last_fullsized":"2023-12-14T18:24:41.940397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:02:24.129455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.843859+0000","last_change":"2023-12-14T18:24:36.178885+0000","last_active":"2023-12-14T18:25:36.843859+0000","last_peered":"2023-12-14T18:25:36.843859+0000","last_clean":"2023-12-14T18:25:36.843859+0000","last_became_active":"2023-12-14T18:24:36.178386+0000","last_became_peered":"2023-12-14T18:24:36.178386+0000","last_unstale":"2023-12-14T18:25:36.843859+0000","last_undegraded":"2023-12-14T18:25:36.843859+0000","last_fullsized":"2023-12-14T18:25:36.843859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:31:44.239186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095247+0000","last_change":"2023-12-14T18:24:36.590037+0000","last_active":"2023-12-14T18:24:41.095247+0000","last_peered":"2023-12-14T18:24:41.095247+0000","last_clean":"2023-12-14T18:24:41.095247+0000","last_became_active":"2023-12-14T18:24:36.589659+0000","last_became_peered":"2023-12-14T18:24:36.589659+0000","last_unstale":"2023-12-14T18:24:41.095247+0000","last_undegraded":"2023-12-14T18:24:41.095247+0000","last_fullsized":"2023-12-14T18:24:41.095247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:13:57.949133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093418+0000","last_change":"2023-12-14T18:24:40.602483+0000","last_active":"2023-12-14T18:24:41.093418+0000","last_peered":"2023-12-14T18:24:41.093418+0000","last_clean":"2023-12-14T18:24:41.093418+0000","last_became_active":"2023-12-14T18:24:40.601989+0000","last_became_peered":"2023-12-14T18:24:40.601989+0000","last_unstale":"2023-12-14T18:24:41.093418+0000","last_undegraded":"2023-12-14T18:24:41.093418+0000","last_fullsized":"2023-12-14T18:24:41.093418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:09.202695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.037079+0000","last_change":"2023-12-14T18:24:36.173350+0000","last_active":"2023-12-14T18:25:00.037079+0000","last_peered":"2023-12-14T18:25:00.037079+0000","last_clean":"2023-12-14T18:25:00.037079+0000","last_became_active":"2023-12-14T18:24:36.172840+0000","last_became_peered":"2023-12-14T18:24:36.172840+0000","last_unstale":"2023-12-14T18:25:00.037079+0000","last_undegraded":"2023-12-14T18:25:00.037079+0000","last_fullsized":"2023-12-14T18:25:00.037079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:17:31.439315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939457+0000","last_change":"2023-12-14T18:24:36.592104+0000","last_active":"2023-12-14T18:24:41.939457+0000","last_peered":"2023-12-14T18:24:41.939457+0000","last_clean":"2023-12-14T18:24:41.939457+0000","last_became_active":"2023-12-14T18:24:36.591568+0000","last_became_peered":"2023-12-14T18:24:36.591568+0000","last_unstale":"2023-12-14T18:24:41.939457+0000","last_undegraded":"2023-12-14T18:24:41.939457+0000","last_fullsized":"2023-12-14T18:24:41.939457+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:30.952113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940589+0000","last_change":"2023-12-14T18:24:40.596043+0000","last_active":"2023-12-14T18:24:41.940589+0000","last_peered":"2023-12-14T18:24:41.940589+0000","last_clean":"2023-12-14T18:24:41.940589+0000","last_became_active":"2023-12-14T18:24:40.595520+0000","last_became_peered":"2023-12-14T18:24:40.595520+0000","last_unstale":"2023-12-14T18:24:41.940589+0000","last_undegraded":"2023-12-14T18:24:41.940589+0000","last_fullsized":"2023-12-14T18:24:41.940589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:45:44.615227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":23,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000728+0000","last_change":"2023-12-14T18:24:36.174184+0000","last_active":"2023-12-14T18:24:41.000728+0000","last_peered":"2023-12-14T18:24:41.000728+0000","last_clean":"2023-12-14T18:24:41.000728+0000","last_became_active":"2023-12-14T18:24:36.173850+0000","last_became_peered":"2023-12-14T18:24:36.173850+0000","last_unstale":"2023-12-14T18:24:41.000728+0000","last_undegraded":"2023-12-14T18:24:41.000728+0000","last_fullsized":"2023-12-14T18:24:41.000728+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:49:58.488522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999206+0000","last_change":"2023-12-14T18:24:36.587777+0000","last_active":"2023-12-14T18:24:40.999206+0000","last_peered":"2023-12-14T18:24:40.999206+0000","last_clean":"2023-12-14T18:24:40.999206+0000","last_became_active":"2023-12-14T18:24:36.587317+0000","last_became_peered":"2023-12-14T18:24:36.587317+0000","last_unstale":"2023-12-14T18:24:40.999206+0000","last_undegraded":"2023-12-14T18:24:40.999206+0000","last_fullsized":"2023-12-14T18:24:40.999206+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:59:49.931215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940400+0000","last_change":"2023-12-14T18:24:40.771913+0000","last_active":"2023-12-14T18:24:41.940400+0000","last_peered":"2023-12-14T18:24:41.940400+0000","last_clean":"2023-12-14T18:24:41.940400+0000","last_became_active":"2023-12-14T18:24:40.771425+0000","last_became_peered":"2023-12-14T18:24:40.771425+0000","last_unstale":"2023-12-14T18:24:41.940400+0000","last_undegraded":"2023-12-14T18:24:41.940400+0000","last_fullsized":"2023-12-14T18:24:41.940400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:17:43.378002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039457+0000","last_change":"2023-12-14T18:24:36.176801+0000","last_active":"2023-12-14T18:25:00.039457+0000","last_peered":"2023-12-14T18:25:00.039457+0000","last_clean":"2023-12-14T18:25:00.039457+0000","last_became_active":"2023-12-14T18:24:36.176001+0000","last_became_peered":"2023-12-14T18:24:36.176001+0000","last_unstale":"2023-12-14T18:25:00.039457+0000","last_undegraded":"2023-12-14T18:25:00.039457+0000","last_fullsized":"2023-12-14T18:25:00.039457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:16:31.583481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845322+0000","last_change":"2023-12-14T18:24:36.592898+0000","last_active":"2023-12-14T18:25:37.845322+0000","last_peered":"2023-12-14T18:25:37.845322+0000","last_clean":"2023-12-14T18:25:37.845322+0000","last_became_active":"2023-12-14T18:24:36.586650+0000","last_became_peered":"2023-12-14T18:24:36.586650+0000","last_unstale":"2023-12-14T18:25:37.845322+0000","last_undegraded":"2023-12-14T18:25:37.845322+0000","last_fullsized":"2023-12-14T18:25:37.845322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:35:59.247019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.845122+0000","last_change":"2023-12-14T18:24:40.599181+0000","last_active":"2023-12-14T18:25:36.845122+0000","last_peered":"2023-12-14T18:25:36.845122+0000","last_clean":"2023-12-14T18:25:36.845122+0000","last_became_active":"2023-12-14T18:24:40.598785+0000","last_became_peered":"2023-12-14T18:24:40.598785+0000","last_unstale":"2023-12-14T18:25:36.845122+0000","last_undegraded":"2023-12-14T18:25:36.845122+0000","last_fullsized":"2023-12-14T18:25:36.845122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:01:59.215376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767511+0000","last_change":"2023-12-14T18:24:36.173644+0000","last_active":"2023-12-14T18:25:00.767511+0000","last_peered":"2023-12-14T18:25:00.767511+0000","last_clean":"2023-12-14T18:25:00.767511+0000","last_became_active":"2023-12-14T18:24:36.173059+0000","last_became_peered":"2023-12-14T18:24:36.173059+0000","last_unstale":"2023-12-14T18:25:00.767511+0000","last_undegraded":"2023-12-14T18:25:00.767511+0000","last_fullsized":"2023-12-14T18:25:00.767511+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:11:21.509881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844525+0000","last_change":"2023-12-14T18:24:36.588761+0000","last_active":"2023-12-14T18:25:37.844525+0000","last_peered":"2023-12-14T18:25:37.844525+0000","last_clean":"2023-12-14T18:25:37.844525+0000","last_became_active":"2023-12-14T18:24:36.586021+0000","last_became_peered":"2023-12-14T18:24:36.586021+0000","last_unstale":"2023-12-14T18:25:37.844525+0000","last_undegraded":"2023-12-14T18:25:37.844525+0000","last_fullsized":"2023-12-14T18:25:37.844525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:45:41.729490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939203+0000","last_change":"2023-12-14T18:24:40.607286+0000","last_active":"2023-12-14T18:24:41.939203+0000","last_peered":"2023-12-14T18:24:41.939203+0000","last_clean":"2023-12-14T18:24:41.939203+0000","last_became_active":"2023-12-14T18:24:40.606925+0000","last_became_peered":"2023-12-14T18:24:40.606925+0000","last_unstale":"2023-12-14T18:24:41.939203+0000","last_undegraded":"2023-12-14T18:24:41.939203+0000","last_fullsized":"2023-12-14T18:24:41.939203+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:47:39.360084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847782+0000","last_change":"2023-12-14T18:24:36.174467+0000","last_active":"2023-12-14T18:25:36.847782+0000","last_peered":"2023-12-14T18:25:36.847782+0000","last_clean":"2023-12-14T18:25:36.847782+0000","last_became_active":"2023-12-14T18:24:36.173920+0000","last_became_peered":"2023-12-14T18:24:36.173920+0000","last_unstale":"2023-12-14T18:25:36.847782+0000","last_undegraded":"2023-12-14T18:25:36.847782+0000","last_fullsized":"2023-12-14T18:25:36.847782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:49:56.456390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846258+0000","last_change":"2023-12-14T18:24:36.597537+0000","last_active":"2023-12-14T18:25:37.846258+0000","last_peered":"2023-12-14T18:25:37.846258+0000","last_clean":"2023-12-14T18:25:37.846258+0000","last_became_active":"2023-12-14T18:24:36.594599+0000","last_became_peered":"2023-12-14T18:24:36.594599+0000","last_unstale":"2023-12-14T18:25:37.846258+0000","last_undegraded":"2023-12-14T18:25:37.846258+0000","last_fullsized":"2023-12-14T18:25:37.846258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:00:18.245087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.846101+0000","last_change":"2023-12-14T18:24:40.597990+0000","last_active":"2023-12-14T18:25:36.846101+0000","last_peered":"2023-12-14T18:25:36.846101+0000","last_clean":"2023-12-14T18:25:36.846101+0000","last_became_active":"2023-12-14T18:24:40.597568+0000","last_became_peered":"2023-12-14T18:24:40.597568+0000","last_unstale":"2023-12-14T18:25:36.846101+0000","last_undegraded":"2023-12-14T18:25:36.846101+0000","last_fullsized":"2023-12-14T18:25:36.846101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:01.498662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940025+0000","last_change":"2023-12-14T18:24:36.173141+0000","last_active":"2023-12-14T18:24:41.940025+0000","last_peered":"2023-12-14T18:24:41.940025+0000","last_clean":"2023-12-14T18:24:41.940025+0000","last_became_active":"2023-12-14T18:24:36.172811+0000","last_became_peered":"2023-12-14T18:24:36.172811+0000","last_unstale":"2023-12-14T18:24:41.940025+0000","last_undegraded":"2023-12-14T18:24:41.940025+0000","last_fullsized":"2023-12-14T18:24:41.940025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:01:41.375107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844789+0000","last_change":"2023-12-14T18:24:36.587333+0000","last_active":"2023-12-14T18:25:37.844789+0000","last_peered":"2023-12-14T18:25:37.844789+0000","last_clean":"2023-12-14T18:25:37.844789+0000","last_became_active":"2023-12-14T18:24:36.586914+0000","last_became_peered":"2023-12-14T18:24:36.586914+0000","last_unstale":"2023-12-14T18:25:37.844789+0000","last_undegraded":"2023-12-14T18:25:37.844789+0000","last_fullsized":"2023-12-14T18:25:37.844789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:48:31.035981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939625+0000","last_change":"2023-12-14T18:24:36.591896+0000","last_active":"2023-12-14T18:24:41.939625+0000","last_peered":"2023-12-14T18:24:41.939625+0000","last_clean":"2023-12-14T18:24:41.939625+0000","last_became_active":"2023-12-14T18:24:36.591097+0000","last_became_peered":"2023-12-14T18:24:36.591097+0000","last_unstale":"2023-12-14T18:24:41.939625+0000","last_undegraded":"2023-12-14T18:24:41.939625+0000","last_fullsized":"2023-12-14T18:24:41.939625+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:14:56.435755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000778+0000","last_change":"2023-12-14T18:24:36.581688+0000","last_active":"2023-12-14T18:24:41.000778+0000","last_peered":"2023-12-14T18:24:41.000778+0000","last_clean":"2023-12-14T18:24:41.000778+0000","last_became_active":"2023-12-14T18:24:36.580660+0000","last_became_peered":"2023-12-14T18:24:36.580660+0000","last_unstale":"2023-12-14T18:24:41.000778+0000","last_undegraded":"2023-12-14T18:24:41.000778+0000","last_fullsized":"2023-12-14T18:24:41.000778+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:13:48.827412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.000650+0000","last_change":"2023-12-14T18:24:36.583025+0000","last_active":"2023-12-14T18:24:41.000650+0000","last_peered":"2023-12-14T18:24:41.000650+0000","last_clean":"2023-12-14T18:24:41.000650+0000","last_became_active":"2023-12-14T18:24:36.582510+0000","last_became_peered":"2023-12-14T18:24:36.582510+0000","last_unstale":"2023-12-14T18:24:41.000650+0000","last_undegraded":"2023-12-14T18:24:41.000650+0000","last_fullsized":"2023-12-14T18:24:41.000650+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:21:38.503753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939373+0000","last_change":"2023-12-14T18:24:36.579383+0000","last_active":"2023-12-14T18:24:41.939373+0000","last_peered":"2023-12-14T18:24:41.939373+0000","last_clean":"2023-12-14T18:24:41.939373+0000","last_became_active":"2023-12-14T18:24:36.579037+0000","last_became_peered":"2023-12-14T18:24:36.579037+0000","last_unstale":"2023-12-14T18:24:41.939373+0000","last_undegraded":"2023-12-14T18:24:41.939373+0000","last_fullsized":"2023-12-14T18:24:41.939373+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:13:32.810036+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940749+0000","last_change":"2023-12-14T18:24:36.582940+0000","last_active":"2023-12-14T18:24:41.940749+0000","last_peered":"2023-12-14T18:24:41.940749+0000","last_clean":"2023-12-14T18:24:41.940749+0000","last_became_active":"2023-12-14T18:24:36.582645+0000","last_became_peered":"2023-12-14T18:24:36.582645+0000","last_unstale":"2023-12-14T18:24:41.940749+0000","last_undegraded":"2023-12-14T18:24:41.940749+0000","last_fullsized":"2023-12-14T18:24:41.940749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:43:46.761797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940015+0000","last_change":"2023-12-14T18:24:36.578613+0000","last_active":"2023-12-14T18:24:41.940015+0000","last_peered":"2023-12-14T18:24:41.940015+0000","last_clean":"2023-12-14T18:24:41.940015+0000","last_became_active":"2023-12-14T18:24:36.578287+0000","last_became_peered":"2023-12-14T18:24:36.578287+0000","last_unstale":"2023-12-14T18:24:41.940015+0000","last_undegraded":"2023-12-14T18:24:41.940015+0000","last_fullsized":"2023-12-14T18:24:41.940015+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:05:43.072668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.001258+0000","last_change":"2023-12-14T18:24:36.575078+0000","last_active":"2023-12-14T18:24:41.001258+0000","last_peered":"2023-12-14T18:24:41.001258+0000","last_clean":"2023-12-14T18:24:41.001258+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:24:41.001258+0000","last_undegraded":"2023-12-14T18:24:41.001258+0000","last_fullsized":"2023-12-14T18:24:41.001258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:55:31.886500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999571+0000","last_change":"2023-12-14T18:24:36.572545+0000","last_active":"2023-12-14T18:24:40.999571+0000","last_peered":"2023-12-14T18:24:40.999571+0000","last_clean":"2023-12-14T18:24:40.999571+0000","last_became_active":"2023-12-14T18:24:36.572235+0000","last_became_peered":"2023-12-14T18:24:36.572235+0000","last_unstale":"2023-12-14T18:24:40.999571+0000","last_undegraded":"2023-12-14T18:24:40.999571+0000","last_fullsized":"2023-12-14T18:24:40.999571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:25:56.081431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.846251+0000","last_change":"2023-12-14T18:24:36.588278+0000","last_active":"2023-12-14T18:25:36.846251+0000","last_peered":"2023-12-14T18:25:36.846251+0000","last_clean":"2023-12-14T18:25:36.846251+0000","last_became_active":"2023-12-14T18:24:36.587605+0000","last_became_peered":"2023-12-14T18:24:36.587605+0000","last_unstale":"2023-12-14T18:25:36.846251+0000","last_undegraded":"2023-12-14T18:25:36.846251+0000","last_fullsized":"2023-12-14T18:25:36.846251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:52.787724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998910+0000","last_change":"2023-12-14T18:24:36.575325+0000","last_active":"2023-12-14T18:24:40.998910+0000","last_peered":"2023-12-14T18:24:40.998910+0000","last_clean":"2023-12-14T18:24:40.998910+0000","last_became_active":"2023-12-14T18:24:36.575016+0000","last_became_peered":"2023-12-14T18:24:36.575016+0000","last_unstale":"2023-12-14T18:24:40.998910+0000","last_undegraded":"2023-12-14T18:24:40.998910+0000","last_fullsized":"2023-12-14T18:24:40.998910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:35:26.394701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.845337+0000","last_change":"2023-12-14T18:24:36.585348+0000","last_active":"2023-12-14T18:25:36.845337+0000","last_peered":"2023-12-14T18:25:36.845337+0000","last_clean":"2023-12-14T18:25:36.845337+0000","last_became_active":"2023-12-14T18:24:36.584995+0000","last_became_peered":"2023-12-14T18:24:36.584995+0000","last_unstale":"2023-12-14T18:25:36.845337+0000","last_undegraded":"2023-12-14T18:25:36.845337+0000","last_fullsized":"2023-12-14T18:25:36.845337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:48:21.622579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939979+0000","last_change":"2023-12-14T18:24:36.583571+0000","last_active":"2023-12-14T18:24:41.939979+0000","last_peered":"2023-12-14T18:24:41.939979+0000","last_clean":"2023-12-14T18:24:41.939979+0000","last_became_active":"2023-12-14T18:24:36.583266+0000","last_became_peered":"2023-12-14T18:24:36.583266+0000","last_unstale":"2023-12-14T18:24:41.939979+0000","last_undegraded":"2023-12-14T18:24:41.939979+0000","last_fullsized":"2023-12-14T18:24:41.939979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:09:22.877080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939741+0000","last_change":"2023-12-14T18:24:36.579754+0000","last_active":"2023-12-14T18:24:41.939741+0000","last_peered":"2023-12-14T18:24:41.939741+0000","last_clean":"2023-12-14T18:24:41.939741+0000","last_became_active":"2023-12-14T18:24:36.579337+0000","last_became_peered":"2023-12-14T18:24:36.579337+0000","last_unstale":"2023-12-14T18:24:41.939741+0000","last_undegraded":"2023-12-14T18:24:41.939741+0000","last_fullsized":"2023-12-14T18:24:41.939741+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:48:45.306297+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212533,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64740,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":236,"num_read_kb":189,"num_write":153,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547222,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6088,"kb_used_data":896,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739064,"statfs":{"total":95995035648,"available":95988801536,"internally_reserved":0,"allocated":917504,"data_stored":458640,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.379},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.57099999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.41399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.436}]},{"osd":5,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.63600000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.68300000000000005}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":9,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.753},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.67100000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.78100000000000003}]}]},{"osd":10,"up_from":67,"seq":287762808858,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":1056768,"data_stored":1106365,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.6690000000000005,"5min":8.6690000000000005,"15min":8.6690000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":9.5459999999999994,"5min":9.5459999999999994,"15min":9.5459999999999994},"last":0.33300000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":9.4380000000000006,"5min":9.4380000000000006,"15min":9.4380000000000006},"last":0.498},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.9470000000000001,"5min":5.9470000000000001,"15min":5.9470000000000001},"last":0.379},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":6.1470000000000002,"5min":6.1470000000000002,"15min":6.1470000000000002},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":9.4830000000000005,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":0.439},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.48399999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.54100000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":6.0309999999999997,"5min":6.0309999999999997,"15min":6.0309999999999997},"last":0.52400000000000002}]}]},{"osd":9,"up_from":62,"seq":266287972380,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6236,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738916,"statfs":{"total":95995035648,"available":95988649984,"internally_reserved":0,"allocated":937984,"data_stored":462972,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42199999999999999,"15min":0.42999999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53500000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.27200000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.73999999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60999999999999999,"15min":0.62},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.83399999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.46800000000000003}]},{"osd":2,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61199999999999999,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.438}]},{"osd":3,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.47199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59199999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.45100000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.879,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.56000000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72999999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.622,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.45900000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80700000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.749},"min":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84799999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.61099999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.81699999999999995,"5min":1.214,"15min":1.214},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.47199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79600000000000004,"5min":1.071,"15min":1.071},"last":0.622}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.53900000000000003}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]}]},{"osd":8,"up_from":55,"seq":236223201313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6356,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738796,"statfs":{"total":95995035648,"available":95988527104,"internally_reserved":0,"allocated":929792,"data_stored":458465,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.501},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.372},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.41299999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47599999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.628},{"interface":"front","average":{"1min":0.754,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71699999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":7,"up_from":48,"seq":206158430244,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6544,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738608,"statfs":{"total":95995035648,"available":95988334592,"internally_reserved":0,"allocated":991232,"data_stored":1063787,"data_compressed":50280,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.434}]},{"osd":1,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.498}]},{"osd":4,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.53600000000000003,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.7830000000000004,"5min":5.7830000000000004,"15min":5.7830000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.29499999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.52000000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77500000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.39900000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.58999999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82699999999999996,"5min":0.879,"15min":0.879},"last":0.68000000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.56399999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.53100000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.50600000000000001}]},{"osd":11,"last update":"Thu Dec 14 18:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68899999999999995}]}]},{"osd":6,"up_from":43,"seq":184683593766,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6340,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738812,"statfs":{"total":95995035648,"available":95988543488,"internally_reserved":0,"allocated":913408,"data_stored":454052,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38500000000000001},"min":{"1min":0.23400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.253,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70399999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.745,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70899999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.46400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.624,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56499999999999995}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.65200000000000002,"15min":0.65700000000000003},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79200000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.501},{"interface":"front","average":{"1min":0.627,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79800000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47199999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.60199999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7364,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737788,"statfs":{"total":95995035648,"available":95987494912,"internally_reserved":0,"allocated":978944,"data_stored":514798,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.46200000000000002},"min":{"1min":0.36099999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.42999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.41299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47799999999999998,"15min":0.46500000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.316,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.40300000000000002},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77600000000000002,"5min":1.542,"15min":1.542},"last":0.34799999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53800000000000003,"15min":0.51900000000000002},"min":{"1min":0.442,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.501}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83299999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.501,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.57899999999999996}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52900000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72299999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":0,"up_from":8,"seq":34359738429,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7196,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737956,"statfs":{"total":95995035648,"available":95987666944,"internally_reserved":0,"allocated":937984,"data_stored":455440,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57599999999999996}]},{"osd":2,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.52500000000000002,"15min":0.497},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54000000000000004,"15min":0.5},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.79000000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.27300000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35499999999999998},"min":{"1min":0.26700000000000002,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.39100000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.38,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51200000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71799999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.67000000000000004}]},{"osd":5,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.51300000000000001,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.626},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.40400000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.48999999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.376},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.51000000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60399999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.74099999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.76500000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.63900000000000001,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":7.2290000000000001,"5min":7.2290000000000001,"15min":7.2290000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.67300000000000004,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.37,"15min":0.37},"max":{"1min":7.3390000000000004,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.47899999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.58499999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.81499999999999995},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.70899999999999996}]}]},{"osd":2,"up_from":18,"seq":77309411381,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7452,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":6464,"kb_avail":93737700,"statfs":{"total":95995035648,"available":95987404800,"internally_reserved":0,"allocated":1003520,"data_stored":516490,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6619136},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.45200000000000001,"15min":0.434},"min":{"1min":0.315,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.64200000000000002,"5min":0.68300000000000005,"15min":0.68300000000000005},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.47099999999999997,"15min":0.44900000000000001},"min":{"1min":0.379,"5min":0.317,"15min":0.317},"max":{"1min":0.65300000000000002,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.44600000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.499,"15min":0.47099999999999997},"min":{"1min":0.35799999999999998,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.51500000000000001,"15min":0.48599999999999999},"min":{"1min":0.39500000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.85599999999999998,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.46600000000000003},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76100000000000001,"5min":1.179,"15min":1.179},"last":0.40400000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.51200000000000001,"15min":0.49199999999999999},"min":{"1min":0.39700000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.84299999999999997,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.53300000000000003,"15min":0.51900000000000002},"min":{"1min":0.437,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:24:37 2023","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.623,"15min":0.625},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.59799999999999998,"15min":0.59799999999999998},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.51100000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.78000000000000003},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.72199999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.67300000000000004}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.72899999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.76900000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68500000000000005}]}]},{"osd":3,"up_from":24,"seq":103079215154,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":925696,"data_stored":454246,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.32800000000000001},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.503},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.248,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.497,"15min":0.47499999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.626}]},{"osd":2,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":1.214,"15min":1.214},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.45300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64100000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66400000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.497,"15min":0.499},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":0.755},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:24:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.628},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.85099999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79900000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.63700000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63700000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986223,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":925696,"data_stored":458654,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48799999999999999,"15min":0.48099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.622},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.48399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.45000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.76500000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44400000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.36899999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.503,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59799999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.51900000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.84499999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822699,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":991232,"data_stored":1050244,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.626,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.501,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.254,"15min":0.254},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.32800000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.628,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.68000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.498},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.997,"15min":0.997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67900000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82299999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.41399999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.58599999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.61799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.50600000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":62528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:25:39.378 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph pg dump --format=json 2023-12-14T18:25:39.688 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:40.200 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:39 smithi093 ceph-mon[42070]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:40.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:39 smithi160 ceph-mon[43840]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:40.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:39 smithi097 ceph-mon[43486]: from='client.15099 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:40.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.786+0000 7f3295f99700 1 -- 172.21.15.93:0/957978881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 msgr2=0x7f3290104ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:40.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.786+0000 7f3295f99700 1 --2- 172.21.15.93:0/957978881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 0x7f3290104ed0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f32840099c0 tx=0x7f3284009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:40.788 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.786+0000 7f3295f99700 1 -- 172.21.15.93:0/957978881 shutdown_connections 2023-12-14T18:25:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.786+0000 7f3295f99700 1 --2- 172.21.15.93:0/957978881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3290105600 0x7f329010d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.786+0000 7f3295f99700 1 --2- 172.21.15.93:0/957978881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3290100560 0x7f3290100930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.786+0000 7f3295f99700 1 --2- 172.21.15.93:0/957978881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 0x7f3290104ed0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.786+0000 7f3295f99700 1 -- 172.21.15.93:0/957978881 >> 172.21.15.93:0/957978881 conn(0x7f32900fbde0 msgr2=0x7f32900fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.786+0000 7f3295f99700 1 -- 172.21.15.93:0/957978881 shutdown_connections 2023-12-14T18:25:40.789 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.786+0000 7f3295f99700 1 -- 172.21.15.93:0/957978881 wait complete. 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.787+0000 7f3295f99700 1 Processor -- start 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.787+0000 7f3295f99700 1 -- start start 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.787+0000 7f3295f99700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3290100560 0x7f32901963c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.787+0000 7f3295f99700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 0x7f3290196900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.787+0000 7f3295f99700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3290105600 0x7f329019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.787+0000 7f3295f99700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329019b380 con 0x7f3290100f00 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.787+0000 7f3295f99700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329019b4f0 con 0x7f3290100560 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.787+0000 7f3295f99700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f329019b6b0 con 0x7f3290105600 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.787+0000 7f328effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 0x7f3290196900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:40.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 0x7f3290196900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47700/0 (socket says 172.21.15.93:47700) 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328effd700 1 -- 172.21.15.93:0/3222106936 learned_addr learned my addr 172.21.15.93:0/3222106936 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328ffff700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3290105600 0x7f329019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328f7fe700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3290100560 0x7f32901963c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328effd700 1 -- 172.21.15.93:0/3222106936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3290105600 msgr2=0x7f329019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328effd700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3290105600 0x7f329019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328effd700 1 -- 172.21.15.93:0/3222106936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3290100560 msgr2=0x7f32901963c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328effd700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3290100560 0x7f32901963c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328effd700 1 -- 172.21.15.93:0/3222106936 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32840095f0 con 0x7f3290100f00 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328effd700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 0x7f3290196900 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f328400b580 tx=0x7f328400f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328cff9700 1 -- 172.21.15.93:0/3222106936 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f328401d070 con 0x7f3290100f00 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3290110330 con 0x7f3290100f00 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328cff9700 1 -- 172.21.15.93:0/3222106936 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3284005510 con 0x7f3290100f00 2023-12-14T18:25:40.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f328cff9700 1 -- 172.21.15.93:0/3222106936 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3284004b30 con 0x7f3290100f00 2023-12-14T18:25:40.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.788+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32901101a0 con 0x7f3290100f00 2023-12-14T18:25:40.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.790+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32901104c0 con 0x7f3290100f00 2023-12-14T18:25:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.790+0000 7f328cff9700 1 -- 172.21.15.93:0/3222106936 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f328400fb80 con 0x7f3290100f00 2023-12-14T18:25:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.791+0000 7f328cff9700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32780a11e0 0x7f32780a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.791+0000 7f328f7fe700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32780a11e0 0x7f32780a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.791+0000 7f328cff9700 1 -- 172.21.15.93:0/3222106936 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f32840cdbf0 con 0x7f3290100f00 2023-12-14T18:25:40.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.791+0000 7f328f7fe700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32780a11e0 0x7f32780a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3280009bf0 tx=0x7f3280009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:40.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.795+0000 7f328cff9700 1 -- 172.21.15.93:0/3222106936 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3284096f60 con 0x7f3290100f00 2023-12-14T18:25:40.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.980+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f32900fcf40 con 0x7f32780a11e0 2023-12-14T18:25:40.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.992+0000 7f328cff9700 1 -- 172.21.15.93:0/3222106936 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+424136 (secure 0 0 0) 0x7f32900fcf40 con 0x7f32780a11e0 2023-12-14T18:25:41.000 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.995+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32780a11e0 msgr2=0x7f32780a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.995+0000 7f3295f99700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32780a11e0 0x7f32780a3690 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f3280009bf0 tx=0x7f3280009260 comp rx=0 tx=0).stop 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.995+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 msgr2=0x7f3290196900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.995+0000 7f3295f99700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 0x7f3290196900 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f328400b580 tx=0x7f328400f690 comp rx=0 tx=0).stop 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.996+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 shutdown_connections 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.996+0000 7f3295f99700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32780a11e0 0x7f32780a3690 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.996+0000 7f3295f99700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3290105600 0x7f329019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.996+0000 7f3295f99700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3290100560 0x7f32901963c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.996+0000 7f3295f99700 1 --2- 172.21.15.93:0/3222106936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3290100f00 0x7f3290196900 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.996+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 >> 172.21.15.93:0/3222106936 conn(0x7f32900fbde0 msgr2=0x7f32900fc830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:41.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.996+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 shutdown_connections 2023-12-14T18:25:41.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:40.996+0000 7f3295f99700 1 -- 172.21.15.93:0/3222106936 wait complete. 2023-12-14T18:25:41.006 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-12-14T18:25:41.150 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:40 smithi093 ceph-mon[42070]: pgmap v258: 129 pgs: 1 activating, 1 peering, 127 active+clean; 640 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:41.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:40 smithi160 ceph-mon[43840]: pgmap v258: 129 pgs: 1 activating, 1 peering, 127 active+clean; 640 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:41.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:40 smithi097 ceph-mon[43486]: pgmap v258: 129 pgs: 1 activating, 1 peering, 127 active+clean; 640 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s rd, 255 B/s wr, 3 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:41.371 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":258,"stamp":"2023-12-14T18:25:39.961295+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655354,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":372,"num_read_kb":308,"num_write":336,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":1180736,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":420,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":79892,"kb_used_data":11316,"kb_used_omap":0,"kb_used_meta":68480,"kb_avail":1124861932,"statfs":{"total":1151940427776,"available":1151858618368,"internally_reserved":0,"allocated":11587584,"data_stored":7519927,"data_compressed":270322,"data_compressed_allocated":364544,"data_compressed_original":2228224,"omap_allocated":0,"internal_metadata":70123520},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":31,"num_read_kb":17,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001919"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844182+0000","last_change":"2023-12-14T18:24:36.587133+0000","last_active":"2023-12-14T18:25:37.844182+0000","last_peered":"2023-12-14T18:25:37.844182+0000","last_clean":"2023-12-14T18:25:37.844182+0000","last_became_active":"2023-12-14T18:24:36.586222+0000","last_became_peered":"2023-12-14T18:24:36.586222+0000","last_unstale":"2023-12-14T18:25:37.844182+0000","last_undegraded":"2023-12-14T18:25:37.844182+0000","last_fullsized":"2023-12-14T18:25:37.844182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:35:18.142196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052570+0000","last_change":"2023-12-14T18:24:36.589027+0000","last_active":"2023-12-14T18:25:38.052570+0000","last_peered":"2023-12-14T18:25:38.052570+0000","last_clean":"2023-12-14T18:25:38.052570+0000","last_became_active":"2023-12-14T18:24:36.588785+0000","last_became_peered":"2023-12-14T18:24:36.588785+0000","last_unstale":"2023-12-14T18:25:38.052570+0000","last_undegraded":"2023-12-14T18:25:38.052570+0000","last_fullsized":"2023-12-14T18:25:38.052570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:10:31.188239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052726+0000","last_change":"2023-12-14T18:24:36.584015+0000","last_active":"2023-12-14T18:25:38.052726+0000","last_peered":"2023-12-14T18:25:38.052726+0000","last_clean":"2023-12-14T18:25:38.052726+0000","last_became_active":"2023-12-14T18:24:36.583735+0000","last_became_peered":"2023-12-14T18:24:36.583735+0000","last_unstale":"2023-12-14T18:25:38.052726+0000","last_undegraded":"2023-12-14T18:25:38.052726+0000","last_fullsized":"2023-12-14T18:25:38.052726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:35:38.893225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940899+0000","last_change":"2023-12-14T18:24:36.584553+0000","last_active":"2023-12-14T18:24:41.940899+0000","last_peered":"2023-12-14T18:24:41.940899+0000","last_clean":"2023-12-14T18:24:41.940899+0000","last_became_active":"2023-12-14T18:24:36.583827+0000","last_became_peered":"2023-12-14T18:24:36.583827+0000","last_unstale":"2023-12-14T18:24:41.940899+0000","last_undegraded":"2023-12-14T18:24:41.940899+0000","last_fullsized":"2023-12-14T18:24:41.940899+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:32.928164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095366+0000","last_change":"2023-12-14T18:24:36.578128+0000","last_active":"2023-12-14T18:24:41.095366+0000","last_peered":"2023-12-14T18:24:41.095366+0000","last_clean":"2023-12-14T18:24:41.095366+0000","last_became_active":"2023-12-14T18:24:36.577784+0000","last_became_peered":"2023-12-14T18:24:36.577784+0000","last_unstale":"2023-12-14T18:24:41.095366+0000","last_undegraded":"2023-12-14T18:24:41.095366+0000","last_fullsized":"2023-12-14T18:24:41.095366+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:56:17.890524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940347+0000","last_change":"2023-12-14T18:24:36.588666+0000","last_active":"2023-12-14T18:24:41.940347+0000","last_peered":"2023-12-14T18:24:41.940347+0000","last_clean":"2023-12-14T18:24:41.940347+0000","last_became_active":"2023-12-14T18:24:36.588291+0000","last_became_peered":"2023-12-14T18:24:36.588291+0000","last_unstale":"2023-12-14T18:24:41.940347+0000","last_undegraded":"2023-12-14T18:24:41.940347+0000","last_fullsized":"2023-12-14T18:24:41.940347+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:10:59.591257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844209+0000","last_change":"2023-12-14T18:24:36.586637+0000","last_active":"2023-12-14T18:25:37.844209+0000","last_peered":"2023-12-14T18:25:37.844209+0000","last_clean":"2023-12-14T18:25:37.844209+0000","last_became_active":"2023-12-14T18:24:36.583055+0000","last_became_peered":"2023-12-14T18:24:36.583055+0000","last_unstale":"2023-12-14T18:25:37.844209+0000","last_undegraded":"2023-12-14T18:25:37.844209+0000","last_fullsized":"2023-12-14T18:25:37.844209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:18:17.900284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998916+0000","last_change":"2023-12-14T18:24:36.580206+0000","last_active":"2023-12-14T18:24:40.998916+0000","last_peered":"2023-12-14T18:24:40.998916+0000","last_clean":"2023-12-14T18:24:40.998916+0000","last_became_active":"2023-12-14T18:24:36.579904+0000","last_became_peered":"2023-12-14T18:24:36.579904+0000","last_unstale":"2023-12-14T18:24:40.998916+0000","last_undegraded":"2023-12-14T18:24:40.998916+0000","last_fullsized":"2023-12-14T18:24:40.998916+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:15.182919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.845055+0000","last_change":"2023-12-14T18:24:36.589890+0000","last_active":"2023-12-14T18:25:36.845055+0000","last_peered":"2023-12-14T18:25:36.845055+0000","last_clean":"2023-12-14T18:25:36.845055+0000","last_became_active":"2023-12-14T18:24:36.588999+0000","last_became_peered":"2023-12-14T18:24:36.588999+0000","last_unstale":"2023-12-14T18:25:36.845055+0000","last_undegraded":"2023-12-14T18:25:36.845055+0000","last_fullsized":"2023-12-14T18:25:36.845055+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:28:51.096009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941211+0000","last_change":"2023-12-14T18:24:36.589565+0000","last_active":"2023-12-14T18:24:41.941211+0000","last_peered":"2023-12-14T18:24:41.941211+0000","last_clean":"2023-12-14T18:24:41.941211+0000","last_became_active":"2023-12-14T18:24:36.588822+0000","last_became_peered":"2023-12-14T18:24:36.588822+0000","last_unstale":"2023-12-14T18:24:41.941211+0000","last_undegraded":"2023-12-14T18:24:41.941211+0000","last_fullsized":"2023-12-14T18:24:41.941211+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:20:30.545830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844272+0000","last_change":"2023-12-14T18:24:36.593208+0000","last_active":"2023-12-14T18:25:37.844272+0000","last_peered":"2023-12-14T18:25:37.844272+0000","last_clean":"2023-12-14T18:25:37.844272+0000","last_became_active":"2023-12-14T18:24:36.586394+0000","last_became_peered":"2023-12-14T18:24:36.586394+0000","last_unstale":"2023-12-14T18:25:37.844272+0000","last_undegraded":"2023-12-14T18:25:37.844272+0000","last_fullsized":"2023-12-14T18:25:37.844272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:22.984010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.055045+0000","last_change":"2023-12-14T18:25:38.054725+0000","last_active":"2023-12-14T18:25:38.055045+0000","last_peered":"2023-12-14T18:25:38.055045+0000","last_clean":"2023-12-14T18:25:38.055045+0000","last_became_active":"2023-12-14T18:25:38.054505+0000","last_became_peered":"2023-12-14T18:25:38.054505+0000","last_unstale":"2023-12-14T18:25:38.055045+0000","last_undegraded":"2023-12-14T18:25:38.055045+0000","last_fullsized":"2023-12-14T18:25:38.055045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:29:35.537082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939911+0000","last_change":"2023-12-14T18:24:36.584971+0000","last_active":"2023-12-14T18:24:41.939911+0000","last_peered":"2023-12-14T18:24:41.939911+0000","last_clean":"2023-12-14T18:24:41.939911+0000","last_became_active":"2023-12-14T18:24:36.584695+0000","last_became_peered":"2023-12-14T18:24:36.584695+0000","last_unstale":"2023-12-14T18:24:41.939911+0000","last_undegraded":"2023-12-14T18:24:41.939911+0000","last_fullsized":"2023-12-14T18:24:41.939911+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:43:12.153321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052278+0000","last_change":"2023-12-14T18:24:36.589692+0000","last_active":"2023-12-14T18:25:38.052278+0000","last_peered":"2023-12-14T18:25:38.052278+0000","last_clean":"2023-12-14T18:25:38.052278+0000","last_became_active":"2023-12-14T18:24:36.589415+0000","last_became_peered":"2023-12-14T18:24:36.589415+0000","last_unstale":"2023-12-14T18:25:38.052278+0000","last_undegraded":"2023-12-14T18:25:38.052278+0000","last_fullsized":"2023-12-14T18:25:38.052278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:34:32.470917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052263+0000","last_change":"2023-12-14T18:24:36.586911+0000","last_active":"2023-12-14T18:25:38.052263+0000","last_peered":"2023-12-14T18:25:38.052263+0000","last_clean":"2023-12-14T18:25:38.052263+0000","last_became_active":"2023-12-14T18:24:36.586600+0000","last_became_peered":"2023-12-14T18:24:36.586600+0000","last_unstale":"2023-12-14T18:25:38.052263+0000","last_undegraded":"2023-12-14T18:25:38.052263+0000","last_fullsized":"2023-12-14T18:25:38.052263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:05:03.670077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.843717+0000","last_change":"2023-12-14T18:24:36.584879+0000","last_active":"2023-12-14T18:25:36.843717+0000","last_peered":"2023-12-14T18:25:36.843717+0000","last_clean":"2023-12-14T18:25:36.843717+0000","last_became_active":"2023-12-14T18:24:36.584566+0000","last_became_peered":"2023-12-14T18:24:36.584566+0000","last_unstale":"2023-12-14T18:25:36.843717+0000","last_undegraded":"2023-12-14T18:25:36.843717+0000","last_fullsized":"2023-12-14T18:25:36.843717+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:04:55.159541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843862+0000","last_change":"2023-12-14T18:24:36.587420+0000","last_active":"2023-12-14T18:25:37.843862+0000","last_peered":"2023-12-14T18:25:37.843862+0000","last_clean":"2023-12-14T18:25:37.843862+0000","last_became_active":"2023-12-14T18:24:36.586732+0000","last_became_peered":"2023-12-14T18:24:36.586732+0000","last_unstale":"2023-12-14T18:25:37.843862+0000","last_undegraded":"2023-12-14T18:25:37.843862+0000","last_fullsized":"2023-12-14T18:25:37.843862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:05:08.359976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842943+0000","last_change":"2023-12-14T18:24:36.583412+0000","last_active":"2023-12-14T18:25:37.842943+0000","last_peered":"2023-12-14T18:25:37.842943+0000","last_clean":"2023-12-14T18:25:37.842943+0000","last_became_active":"2023-12-14T18:24:36.583022+0000","last_became_peered":"2023-12-14T18:24:36.583022+0000","last_unstale":"2023-12-14T18:25:37.842943+0000","last_undegraded":"2023-12-14T18:25:37.842943+0000","last_fullsized":"2023-12-14T18:25:37.842943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:07:57.550138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841957+0000","last_change":"2023-12-14T18:24:36.575232+0000","last_active":"2023-12-14T18:25:37.841957+0000","last_peered":"2023-12-14T18:25:37.841957+0000","last_clean":"2023-12-14T18:25:37.841957+0000","last_became_active":"2023-12-14T18:24:36.574573+0000","last_became_peered":"2023-12-14T18:24:36.574573+0000","last_unstale":"2023-12-14T18:25:37.841957+0000","last_undegraded":"2023-12-14T18:25:37.841957+0000","last_fullsized":"2023-12-14T18:25:37.841957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:16:28.555112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840943+0000","last_change":"2023-12-14T18:24:40.605077+0000","last_active":"2023-12-14T18:25:37.840943+0000","last_peered":"2023-12-14T18:25:37.840943+0000","last_clean":"2023-12-14T18:25:37.840943+0000","last_became_active":"2023-12-14T18:24:40.604646+0000","last_became_peered":"2023-12-14T18:24:40.604646+0000","last_unstale":"2023-12-14T18:25:37.840943+0000","last_undegraded":"2023-12-14T18:25:37.840943+0000","last_fullsized":"2023-12-14T18:25:37.840943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:59:21.706575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842596+0000","last_change":"2023-12-14T18:24:36.177449+0000","last_active":"2023-12-14T18:25:37.842596+0000","last_peered":"2023-12-14T18:25:37.842596+0000","last_clean":"2023-12-14T18:25:37.842596+0000","last_became_active":"2023-12-14T18:24:36.177114+0000","last_became_peered":"2023-12-14T18:24:36.177114+0000","last_unstale":"2023-12-14T18:25:37.842596+0000","last_undegraded":"2023-12-14T18:25:37.842596+0000","last_fullsized":"2023-12-14T18:25:37.842596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:20.038616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999464+0000","last_change":"2023-12-14T18:24:36.574204+0000","last_active":"2023-12-14T18:24:40.999464+0000","last_peered":"2023-12-14T18:24:40.999464+0000","last_clean":"2023-12-14T18:24:40.999464+0000","last_became_active":"2023-12-14T18:24:36.573923+0000","last_became_peered":"2023-12-14T18:24:36.573923+0000","last_unstale":"2023-12-14T18:24:40.999464+0000","last_undegraded":"2023-12-14T18:24:40.999464+0000","last_fullsized":"2023-12-14T18:24:40.999464+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:20:32.637706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939617+0000","last_change":"2023-12-14T18:24:40.600254+0000","last_active":"2023-12-14T18:24:41.939617+0000","last_peered":"2023-12-14T18:24:41.939617+0000","last_clean":"2023-12-14T18:24:41.939617+0000","last_became_active":"2023-12-14T18:24:40.599894+0000","last_became_peered":"2023-12-14T18:24:40.599894+0000","last_unstale":"2023-12-14T18:24:41.939617+0000","last_undegraded":"2023-12-14T18:24:41.939617+0000","last_fullsized":"2023-12-14T18:24:41.939617+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:07:00.116137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999521+0000","last_change":"2023-12-14T18:24:36.170109+0000","last_active":"2023-12-14T18:24:40.999521+0000","last_peered":"2023-12-14T18:24:40.999521+0000","last_clean":"2023-12-14T18:24:40.999521+0000","last_became_active":"2023-12-14T18:24:36.169629+0000","last_became_peered":"2023-12-14T18:24:36.169629+0000","last_unstale":"2023-12-14T18:24:40.999521+0000","last_undegraded":"2023-12-14T18:24:40.999521+0000","last_fullsized":"2023-12-14T18:24:40.999521+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:41:16.457493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052504+0000","last_change":"2023-12-14T18:24:36.582757+0000","last_active":"2023-12-14T18:25:38.052504+0000","last_peered":"2023-12-14T18:25:38.052504+0000","last_clean":"2023-12-14T18:25:38.052504+0000","last_became_active":"2023-12-14T18:24:36.582457+0000","last_became_peered":"2023-12-14T18:24:36.582457+0000","last_unstale":"2023-12-14T18:25:38.052504+0000","last_undegraded":"2023-12-14T18:25:38.052504+0000","last_fullsized":"2023-12-14T18:25:38.052504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:35:13.993694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940384+0000","last_change":"2023-12-14T18:24:40.599695+0000","last_active":"2023-12-14T18:24:41.940384+0000","last_peered":"2023-12-14T18:24:41.940384+0000","last_clean":"2023-12-14T18:24:41.940384+0000","last_became_active":"2023-12-14T18:24:40.599239+0000","last_became_peered":"2023-12-14T18:24:40.599239+0000","last_unstale":"2023-12-14T18:24:41.940384+0000","last_undegraded":"2023-12-14T18:24:41.940384+0000","last_fullsized":"2023-12-14T18:24:41.940384+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:40.237886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.766757+0000","last_change":"2023-12-14T18:24:36.177946+0000","last_active":"2023-12-14T18:25:00.766757+0000","last_peered":"2023-12-14T18:25:00.766757+0000","last_clean":"2023-12-14T18:25:00.766757+0000","last_became_active":"2023-12-14T18:24:36.177480+0000","last_became_peered":"2023-12-14T18:24:36.177480+0000","last_unstale":"2023-12-14T18:25:00.766757+0000","last_undegraded":"2023-12-14T18:25:00.766757+0000","last_fullsized":"2023-12-14T18:25:00.766757+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:42:02.447975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.844376+0000","last_change":"2023-12-14T18:24:36.587951+0000","last_active":"2023-12-14T18:25:36.844376+0000","last_peered":"2023-12-14T18:25:36.844376+0000","last_clean":"2023-12-14T18:25:36.844376+0000","last_became_active":"2023-12-14T18:24:36.587539+0000","last_became_peered":"2023-12-14T18:24:36.587539+0000","last_unstale":"2023-12-14T18:25:36.844376+0000","last_undegraded":"2023-12-14T18:25:36.844376+0000","last_fullsized":"2023-12-14T18:25:36.844376+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:21:13.030894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941747+0000","last_change":"2023-12-14T18:24:40.600008+0000","last_active":"2023-12-14T18:24:41.941747+0000","last_peered":"2023-12-14T18:24:41.941747+0000","last_clean":"2023-12-14T18:24:41.941747+0000","last_became_active":"2023-12-14T18:24:40.599596+0000","last_became_peered":"2023-12-14T18:24:40.599596+0000","last_unstale":"2023-12-14T18:24:41.941747+0000","last_undegraded":"2023-12-14T18:24:41.941747+0000","last_fullsized":"2023-12-14T18:24:41.941747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:10:10.770024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095629+0000","last_change":"2023-12-14T18:24:36.176569+0000","last_active":"2023-12-14T18:24:41.095629+0000","last_peered":"2023-12-14T18:24:41.095629+0000","last_clean":"2023-12-14T18:24:41.095629+0000","last_became_active":"2023-12-14T18:24:36.176043+0000","last_became_peered":"2023-12-14T18:24:36.176043+0000","last_unstale":"2023-12-14T18:24:41.095629+0000","last_undegraded":"2023-12-14T18:24:41.095629+0000","last_fullsized":"2023-12-14T18:24:41.095629+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:10:44.020419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847627+0000","last_change":"2023-12-14T18:24:36.593761+0000","last_active":"2023-12-14T18:25:36.847627+0000","last_peered":"2023-12-14T18:25:36.847627+0000","last_clean":"2023-12-14T18:25:36.847627+0000","last_became_active":"2023-12-14T18:24:36.593452+0000","last_became_peered":"2023-12-14T18:24:36.593452+0000","last_unstale":"2023-12-14T18:25:36.847627+0000","last_undegraded":"2023-12-14T18:25:36.847627+0000","last_fullsized":"2023-12-14T18:25:36.847627+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:36:25.313521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841661+0000","last_change":"2023-12-14T18:24:40.598064+0000","last_active":"2023-12-14T18:25:37.841661+0000","last_peered":"2023-12-14T18:25:37.841661+0000","last_clean":"2023-12-14T18:25:37.841661+0000","last_became_active":"2023-12-14T18:24:40.597672+0000","last_became_peered":"2023-12-14T18:24:40.597672+0000","last_unstale":"2023-12-14T18:25:37.841661+0000","last_undegraded":"2023-12-14T18:25:37.841661+0000","last_fullsized":"2023-12-14T18:25:37.841661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:19:47.256702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":69,"reported_epoch":84,"state":"peering","last_fresh":"2023-12-14T18:25:36.813236+0000","last_change":"2023-12-14T18:25:36.845980+0000","last_active":"2023-12-14T18:25:36.845980+0000","last_peered":"2023-12-14T18:25:36.813236+0000","last_clean":"2023-12-14T18:25:36.813236+0000","last_became_active":"2023-12-14T18:24:36.175965+0000","last_became_peered":"2023-12-14T18:24:36.175965+0000","last_unstale":"2023-12-14T18:25:36.813236+0000","last_undegraded":"2023-12-14T18:25:36.813236+0000","last_fullsized":"2023-12-14T18:25:36.813236+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[2,4,7],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095492+0000","last_change":"2023-12-14T18:24:36.583687+0000","last_active":"2023-12-14T18:24:41.095492+0000","last_peered":"2023-12-14T18:24:41.095492+0000","last_clean":"2023-12-14T18:24:41.095492+0000","last_became_active":"2023-12-14T18:24:36.583047+0000","last_became_peered":"2023-12-14T18:24:36.583047+0000","last_unstale":"2023-12-14T18:24:41.095492+0000","last_undegraded":"2023-12-14T18:24:41.095492+0000","last_fullsized":"2023-12-14T18:24:41.095492+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:26:10.095059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841353+0000","last_change":"2023-12-14T18:24:40.605217+0000","last_active":"2023-12-14T18:25:37.841353+0000","last_peered":"2023-12-14T18:25:37.841353+0000","last_clean":"2023-12-14T18:25:37.841353+0000","last_became_active":"2023-12-14T18:24:40.604807+0000","last_became_peered":"2023-12-14T18:24:40.604807+0000","last_unstale":"2023-12-14T18:25:37.841353+0000","last_undegraded":"2023-12-14T18:25:37.841353+0000","last_fullsized":"2023-12-14T18:25:37.841353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:00:15.863206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767627+0000","last_change":"2023-12-14T18:24:36.174875+0000","last_active":"2023-12-14T18:25:00.767627+0000","last_peered":"2023-12-14T18:25:00.767627+0000","last_clean":"2023-12-14T18:25:00.767627+0000","last_became_active":"2023-12-14T18:24:36.173984+0000","last_became_peered":"2023-12-14T18:24:36.173984+0000","last_unstale":"2023-12-14T18:25:00.767627+0000","last_undegraded":"2023-12-14T18:25:00.767627+0000","last_fullsized":"2023-12-14T18:25:00.767627+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:24:27.753878+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847649+0000","last_change":"2023-12-14T18:24:36.580563+0000","last_active":"2023-12-14T18:25:36.847649+0000","last_peered":"2023-12-14T18:25:36.847649+0000","last_clean":"2023-12-14T18:25:36.847649+0000","last_became_active":"2023-12-14T18:24:36.580196+0000","last_became_peered":"2023-12-14T18:24:36.580196+0000","last_unstale":"2023-12-14T18:25:36.847649+0000","last_undegraded":"2023-12-14T18:25:36.847649+0000","last_fullsized":"2023-12-14T18:25:36.847649+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:22:54.569465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843700+0000","last_change":"2023-12-14T18:24:40.601223+0000","last_active":"2023-12-14T18:25:37.843700+0000","last_peered":"2023-12-14T18:25:37.843700+0000","last_clean":"2023-12-14T18:25:37.843700+0000","last_became_active":"2023-12-14T18:24:40.600795+0000","last_became_peered":"2023-12-14T18:24:40.600795+0000","last_unstale":"2023-12-14T18:25:37.843700+0000","last_undegraded":"2023-12-14T18:25:37.843700+0000","last_fullsized":"2023-12-14T18:25:37.843700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:31.442312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767406+0000","last_change":"2023-12-14T18:24:36.173600+0000","last_active":"2023-12-14T18:25:00.767406+0000","last_peered":"2023-12-14T18:25:00.767406+0000","last_clean":"2023-12-14T18:25:00.767406+0000","last_became_active":"2023-12-14T18:24:36.173272+0000","last_became_peered":"2023-12-14T18:24:36.173272+0000","last_unstale":"2023-12-14T18:25:00.767406+0000","last_undegraded":"2023-12-14T18:25:00.767406+0000","last_fullsized":"2023-12-14T18:25:00.767406+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:29:23.589714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998814+0000","last_change":"2023-12-14T18:24:36.575060+0000","last_active":"2023-12-14T18:24:40.998814+0000","last_peered":"2023-12-14T18:24:40.998814+0000","last_clean":"2023-12-14T18:24:40.998814+0000","last_became_active":"2023-12-14T18:24:36.574749+0000","last_became_peered":"2023-12-14T18:24:36.574749+0000","last_unstale":"2023-12-14T18:24:40.998814+0000","last_undegraded":"2023-12-14T18:24:40.998814+0000","last_fullsized":"2023-12-14T18:24:40.998814+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:44:44.163762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052543+0000","last_change":"2023-12-14T18:24:40.605431+0000","last_active":"2023-12-14T18:25:38.052543+0000","last_peered":"2023-12-14T18:25:38.052543+0000","last_clean":"2023-12-14T18:25:38.052543+0000","last_became_active":"2023-12-14T18:24:40.604951+0000","last_became_peered":"2023-12-14T18:24:40.604951+0000","last_unstale":"2023-12-14T18:25:38.052543+0000","last_undegraded":"2023-12-14T18:25:38.052543+0000","last_fullsized":"2023-12-14T18:25:38.052543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:52:55.859566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":25,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.846899+0000","last_change":"2023-12-14T18:24:36.175493+0000","last_active":"2023-12-14T18:25:36.846899+0000","last_peered":"2023-12-14T18:25:36.846899+0000","last_clean":"2023-12-14T18:25:36.846899+0000","last_became_active":"2023-12-14T18:24:36.175205+0000","last_became_peered":"2023-12-14T18:24:36.175205+0000","last_unstale":"2023-12-14T18:25:36.846899+0000","last_undegraded":"2023-12-14T18:25:36.846899+0000","last_fullsized":"2023-12-14T18:25:36.846899+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:06:22.692820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842088+0000","last_change":"2023-12-14T18:24:36.587218+0000","last_active":"2023-12-14T18:25:37.842088+0000","last_peered":"2023-12-14T18:25:37.842088+0000","last_clean":"2023-12-14T18:25:37.842088+0000","last_became_active":"2023-12-14T18:24:36.586910+0000","last_became_peered":"2023-12-14T18:24:36.586910+0000","last_unstale":"2023-12-14T18:25:37.842088+0000","last_undegraded":"2023-12-14T18:25:37.842088+0000","last_fullsized":"2023-12-14T18:25:37.842088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:27.244227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999151+0000","last_change":"2023-12-14T18:24:40.598409+0000","last_active":"2023-12-14T18:24:40.999151+0000","last_peered":"2023-12-14T18:24:40.999151+0000","last_clean":"2023-12-14T18:24:40.999151+0000","last_became_active":"2023-12-14T18:24:40.598011+0000","last_became_peered":"2023-12-14T18:24:40.598011+0000","last_unstale":"2023-12-14T18:24:40.999151+0000","last_undegraded":"2023-12-14T18:24:40.999151+0000","last_fullsized":"2023-12-14T18:24:40.999151+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:12:30.946044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:34.810413+0000","last_change":"2023-12-14T18:24:36.172277+0000","last_active":"2023-12-14T18:25:34.810413+0000","last_peered":"2023-12-14T18:25:34.810413+0000","last_clean":"2023-12-14T18:25:34.810413+0000","last_became_active":"2023-12-14T18:24:36.171907+0000","last_became_peered":"2023-12-14T18:24:36.171907+0000","last_unstale":"2023-12-14T18:25:34.810413+0000","last_undegraded":"2023-12-14T18:25:34.810413+0000","last_fullsized":"2023-12-14T18:25:34.810413+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:45:12.002124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":36,"num_read_kb":36,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.844992+0000","last_change":"2023-12-14T18:24:36.584783+0000","last_active":"2023-12-14T18:25:36.844992+0000","last_peered":"2023-12-14T18:25:36.844992+0000","last_clean":"2023-12-14T18:25:36.844992+0000","last_became_active":"2023-12-14T18:24:36.584424+0000","last_became_peered":"2023-12-14T18:24:36.584424+0000","last_unstale":"2023-12-14T18:25:36.844992+0000","last_undegraded":"2023-12-14T18:25:36.844992+0000","last_fullsized":"2023-12-14T18:25:36.844992+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:54:02.928872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841959+0000","last_change":"2023-12-14T18:24:40.603176+0000","last_active":"2023-12-14T18:25:37.841959+0000","last_peered":"2023-12-14T18:25:37.841959+0000","last_clean":"2023-12-14T18:25:37.841959+0000","last_became_active":"2023-12-14T18:24:40.602868+0000","last_became_peered":"2023-12-14T18:24:40.602868+0000","last_unstale":"2023-12-14T18:25:37.841959+0000","last_undegraded":"2023-12-14T18:25:37.841959+0000","last_fullsized":"2023-12-14T18:25:37.841959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:15:57.475854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844517+0000","last_change":"2023-12-14T18:24:36.177638+0000","last_active":"2023-12-14T18:25:37.844517+0000","last_peered":"2023-12-14T18:25:37.844517+0000","last_clean":"2023-12-14T18:25:37.844517+0000","last_became_active":"2023-12-14T18:24:36.177335+0000","last_became_peered":"2023-12-14T18:24:36.177335+0000","last_unstale":"2023-12-14T18:25:37.844517+0000","last_undegraded":"2023-12-14T18:25:37.844517+0000","last_fullsized":"2023-12-14T18:25:37.844517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:29:24.204595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052948+0000","last_change":"2023-12-14T18:24:36.577508+0000","last_active":"2023-12-14T18:25:38.052948+0000","last_peered":"2023-12-14T18:25:38.052948+0000","last_clean":"2023-12-14T18:25:38.052948+0000","last_became_active":"2023-12-14T18:24:36.577196+0000","last_became_peered":"2023-12-14T18:24:36.577196+0000","last_unstale":"2023-12-14T18:25:38.052948+0000","last_undegraded":"2023-12-14T18:25:38.052948+0000","last_fullsized":"2023-12-14T18:25:38.052948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:55:24.227563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.941240+0000","last_change":"2023-12-14T18:24:40.594816+0000","last_active":"2023-12-14T18:24:41.941240+0000","last_peered":"2023-12-14T18:24:41.941240+0000","last_clean":"2023-12-14T18:24:41.941240+0000","last_became_active":"2023-12-14T18:24:40.594371+0000","last_became_peered":"2023-12-14T18:24:40.594371+0000","last_unstale":"2023-12-14T18:24:41.941240+0000","last_undegraded":"2023-12-14T18:24:41.941240+0000","last_fullsized":"2023-12-14T18:24:41.941240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:40:46.343094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":29,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847519+0000","last_change":"2023-12-14T18:24:36.174238+0000","last_active":"2023-12-14T18:25:36.847519+0000","last_peered":"2023-12-14T18:25:36.847519+0000","last_clean":"2023-12-14T18:25:36.847519+0000","last_became_active":"2023-12-14T18:24:36.173373+0000","last_became_peered":"2023-12-14T18:24:36.173373+0000","last_unstale":"2023-12-14T18:25:36.847519+0000","last_undegraded":"2023-12-14T18:25:36.847519+0000","last_fullsized":"2023-12-14T18:25:36.847519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:24:20.899377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.844664+0000","last_change":"2023-12-14T18:24:36.588589+0000","last_active":"2023-12-14T18:25:36.844664+0000","last_peered":"2023-12-14T18:25:36.844664+0000","last_clean":"2023-12-14T18:25:36.844664+0000","last_became_active":"2023-12-14T18:24:36.588269+0000","last_became_peered":"2023-12-14T18:24:36.588269+0000","last_unstale":"2023-12-14T18:25:36.844664+0000","last_undegraded":"2023-12-14T18:25:36.844664+0000","last_fullsized":"2023-12-14T18:25:36.844664+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:23.993313+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842244+0000","last_change":"2023-12-14T18:24:40.601333+0000","last_active":"2023-12-14T18:25:37.842244+0000","last_peered":"2023-12-14T18:25:37.842244+0000","last_clean":"2023-12-14T18:25:37.842244+0000","last_became_active":"2023-12-14T18:24:40.600966+0000","last_became_peered":"2023-12-14T18:24:40.600966+0000","last_unstale":"2023-12-14T18:25:37.842244+0000","last_undegraded":"2023-12-14T18:25:37.842244+0000","last_fullsized":"2023-12-14T18:25:37.842244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:30:57.057783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":133,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:35.811430+0000","last_change":"2023-12-14T18:24:36.177299+0000","last_active":"2023-12-14T18:25:35.811430+0000","last_peered":"2023-12-14T18:25:35.811430+0000","last_clean":"2023-12-14T18:25:35.811430+0000","last_became_active":"2023-12-14T18:24:36.176971+0000","last_became_peered":"2023-12-14T18:24:36.176971+0000","last_unstale":"2023-12-14T18:25:35.811430+0000","last_undegraded":"2023-12-14T18:25:35.811430+0000","last_fullsized":"2023-12-14T18:25:35.811430+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:02:46.083153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60836,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":80,"num_read_kb":99,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095663+0000","last_change":"2023-12-14T18:24:36.586315+0000","last_active":"2023-12-14T18:24:41.095663+0000","last_peered":"2023-12-14T18:24:41.095663+0000","last_clean":"2023-12-14T18:24:41.095663+0000","last_became_active":"2023-12-14T18:24:36.586000+0000","last_became_peered":"2023-12-14T18:24:36.586000+0000","last_unstale":"2023-12-14T18:24:41.095663+0000","last_undegraded":"2023-12-14T18:24:41.095663+0000","last_fullsized":"2023-12-14T18:24:41.095663+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:19:55.558718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847904+0000","last_change":"2023-12-14T18:24:40.596973+0000","last_active":"2023-12-14T18:25:36.847904+0000","last_peered":"2023-12-14T18:25:36.847904+0000","last_clean":"2023-12-14T18:25:36.847904+0000","last_became_active":"2023-12-14T18:24:40.596355+0000","last_became_peered":"2023-12-14T18:24:40.596355+0000","last_unstale":"2023-12-14T18:25:36.847904+0000","last_undegraded":"2023-12-14T18:25:36.847904+0000","last_fullsized":"2023-12-14T18:25:36.847904+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:42:06.172578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840994+0000","last_change":"2023-12-14T18:24:36.175287+0000","last_active":"2023-12-14T18:25:37.840994+0000","last_peered":"2023-12-14T18:25:37.840994+0000","last_clean":"2023-12-14T18:25:37.840994+0000","last_became_active":"2023-12-14T18:24:36.174952+0000","last_became_peered":"2023-12-14T18:24:36.174952+0000","last_unstale":"2023-12-14T18:25:37.840994+0000","last_undegraded":"2023-12-14T18:25:37.840994+0000","last_fullsized":"2023-12-14T18:25:37.840994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:56:55.084380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940588+0000","last_change":"2023-12-14T18:24:36.582314+0000","last_active":"2023-12-14T18:24:41.940588+0000","last_peered":"2023-12-14T18:24:41.940588+0000","last_clean":"2023-12-14T18:24:41.940588+0000","last_became_active":"2023-12-14T18:24:36.581963+0000","last_became_peered":"2023-12-14T18:24:36.581963+0000","last_unstale":"2023-12-14T18:24:41.940588+0000","last_undegraded":"2023-12-14T18:24:41.940588+0000","last_fullsized":"2023-12-14T18:24:41.940588+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:02:38.477328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843110+0000","last_change":"2023-12-14T18:24:40.599979+0000","last_active":"2023-12-14T18:25:37.843110+0000","last_peered":"2023-12-14T18:25:37.843110+0000","last_clean":"2023-12-14T18:25:37.843110+0000","last_became_active":"2023-12-14T18:24:40.599258+0000","last_became_peered":"2023-12-14T18:24:40.599258+0000","last_unstale":"2023-12-14T18:25:37.843110+0000","last_undegraded":"2023-12-14T18:25:37.843110+0000","last_fullsized":"2023-12-14T18:25:37.843110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:16:48.809775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":27,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847541+0000","last_change":"2023-12-14T18:24:36.175768+0000","last_active":"2023-12-14T18:25:36.847541+0000","last_peered":"2023-12-14T18:25:36.847541+0000","last_clean":"2023-12-14T18:25:36.847541+0000","last_became_active":"2023-12-14T18:24:36.174973+0000","last_became_peered":"2023-12-14T18:24:36.174973+0000","last_unstale":"2023-12-14T18:25:36.847541+0000","last_undegraded":"2023-12-14T18:25:36.847541+0000","last_fullsized":"2023-12-14T18:25:36.847541+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:55:13.193815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844716+0000","last_change":"2023-12-14T18:24:36.591856+0000","last_active":"2023-12-14T18:25:37.844716+0000","last_peered":"2023-12-14T18:25:37.844716+0000","last_clean":"2023-12-14T18:25:37.844716+0000","last_became_active":"2023-12-14T18:24:36.591554+0000","last_became_peered":"2023-12-14T18:24:36.591554+0000","last_unstale":"2023-12-14T18:25:37.844716+0000","last_undegraded":"2023-12-14T18:25:37.844716+0000","last_fullsized":"2023-12-14T18:25:37.844716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:07:38.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842256+0000","last_change":"2023-12-14T18:24:40.603794+0000","last_active":"2023-12-14T18:25:37.842256+0000","last_peered":"2023-12-14T18:25:37.842256+0000","last_clean":"2023-12-14T18:25:37.842256+0000","last_became_active":"2023-12-14T18:24:40.603311+0000","last_became_peered":"2023-12-14T18:24:40.603311+0000","last_unstale":"2023-12-14T18:25:37.842256+0000","last_undegraded":"2023-12-14T18:25:37.842256+0000","last_fullsized":"2023-12-14T18:25:37.842256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:27:38.742031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:33.832001+0000","last_change":"2023-12-14T18:24:36.177411+0000","last_active":"2023-12-14T18:25:33.832001+0000","last_peered":"2023-12-14T18:25:33.832001+0000","last_clean":"2023-12-14T18:25:33.832001+0000","last_became_active":"2023-12-14T18:24:36.177047+0000","last_became_peered":"2023-12-14T18:24:36.177047+0000","last_unstale":"2023-12-14T18:25:33.832001+0000","last_undegraded":"2023-12-14T18:25:33.832001+0000","last_fullsized":"2023-12-14T18:25:33.832001+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:58:22.871429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":37,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"activating","last_fresh":"2023-12-14T18:25:37.844035+0000","last_change":"2023-12-14T18:25:37.844035+0000","last_active":"2023-12-14T18:25:36.841095+0000","last_peered":"2023-12-14T18:24:41.939941+0000","last_clean":"2023-12-14T18:24:41.939941+0000","last_became_active":"2023-12-14T18:24:36.596900+0000","last_became_peered":"2023-12-14T18:24:36.596900+0000","last_unstale":"2023-12-14T18:25:37.844035+0000","last_undegraded":"2023-12-14T18:25:37.844035+0000","last_fullsized":"2023-12-14T18:25:37.844035+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:19:00.666432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":["7","5","9","10"],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940190+0000","last_change":"2023-12-14T18:24:40.607042+0000","last_active":"2023-12-14T18:24:41.940190+0000","last_peered":"2023-12-14T18:24:41.940190+0000","last_clean":"2023-12-14T18:24:41.940190+0000","last_became_active":"2023-12-14T18:24:40.606563+0000","last_became_peered":"2023-12-14T18:24:40.606563+0000","last_unstale":"2023-12-14T18:24:41.940190+0000","last_undegraded":"2023-12-14T18:24:41.940190+0000","last_fullsized":"2023-12-14T18:24:41.940190+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:31:29.839031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":38,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.769004+0000","last_change":"2023-12-14T18:24:36.171994+0000","last_active":"2023-12-14T18:25:00.769004+0000","last_peered":"2023-12-14T18:25:00.769004+0000","last_clean":"2023-12-14T18:25:00.769004+0000","last_became_active":"2023-12-14T18:24:36.171661+0000","last_became_peered":"2023-12-14T18:24:36.171661+0000","last_unstale":"2023-12-14T18:25:00.769004+0000","last_undegraded":"2023-12-14T18:25:00.769004+0000","last_fullsized":"2023-12-14T18:25:00.769004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:33:33.313381+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847106+0000","last_change":"2023-12-14T18:24:36.577818+0000","last_active":"2023-12-14T18:25:36.847106+0000","last_peered":"2023-12-14T18:25:36.847106+0000","last_clean":"2023-12-14T18:25:36.847106+0000","last_became_active":"2023-12-14T18:24:36.577356+0000","last_became_peered":"2023-12-14T18:24:36.577356+0000","last_unstale":"2023-12-14T18:25:36.847106+0000","last_undegraded":"2023-12-14T18:25:36.847106+0000","last_fullsized":"2023-12-14T18:25:36.847106+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:19:22.337928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847031+0000","last_change":"2023-12-14T18:24:40.601995+0000","last_active":"2023-12-14T18:25:36.847031+0000","last_peered":"2023-12-14T18:25:36.847031+0000","last_clean":"2023-12-14T18:25:36.847031+0000","last_became_active":"2023-12-14T18:24:40.601571+0000","last_became_peered":"2023-12-14T18:24:40.601571+0000","last_unstale":"2023-12-14T18:25:36.847031+0000","last_undegraded":"2023-12-14T18:25:36.847031+0000","last_fullsized":"2023-12-14T18:25:36.847031+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:40:59.842261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.756794+0000","last_change":"2023-12-14T18:24:36.168661+0000","last_active":"2023-12-14T18:25:00.756794+0000","last_peered":"2023-12-14T18:25:00.756794+0000","last_clean":"2023-12-14T18:25:00.756794+0000","last_became_active":"2023-12-14T18:24:36.168299+0000","last_became_peered":"2023-12-14T18:24:36.168299+0000","last_unstale":"2023-12-14T18:25:00.756794+0000","last_undegraded":"2023-12-14T18:25:00.756794+0000","last_fullsized":"2023-12-14T18:25:00.756794+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:19:55.288902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843080+0000","last_change":"2023-12-14T18:24:36.578417+0000","last_active":"2023-12-14T18:25:37.843080+0000","last_peered":"2023-12-14T18:25:37.843080+0000","last_clean":"2023-12-14T18:25:37.843080+0000","last_became_active":"2023-12-14T18:24:36.578080+0000","last_became_peered":"2023-12-14T18:24:36.578080+0000","last_unstale":"2023-12-14T18:25:37.843080+0000","last_undegraded":"2023-12-14T18:25:37.843080+0000","last_fullsized":"2023-12-14T18:25:37.843080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:55:52.265641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095510+0000","last_change":"2023-12-14T18:24:40.771866+0000","last_active":"2023-12-14T18:24:41.095510+0000","last_peered":"2023-12-14T18:24:41.095510+0000","last_clean":"2023-12-14T18:24:41.095510+0000","last_became_active":"2023-12-14T18:24:40.771324+0000","last_became_peered":"2023-12-14T18:24:40.771324+0000","last_unstale":"2023-12-14T18:24:41.095510+0000","last_undegraded":"2023-12-14T18:24:41.095510+0000","last_fullsized":"2023-12-14T18:24:41.095510+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:20:19.058538+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842458+0000","last_change":"2023-12-14T18:24:36.170489+0000","last_active":"2023-12-14T18:25:37.842458+0000","last_peered":"2023-12-14T18:25:37.842458+0000","last_clean":"2023-12-14T18:25:37.842458+0000","last_became_active":"2023-12-14T18:24:36.170161+0000","last_became_peered":"2023-12-14T18:24:36.170161+0000","last_unstale":"2023-12-14T18:25:37.842458+0000","last_undegraded":"2023-12-14T18:25:37.842458+0000","last_fullsized":"2023-12-14T18:25:37.842458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:49:25.035687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940761+0000","last_change":"2023-12-14T18:24:36.591094+0000","last_active":"2023-12-14T18:24:41.940761+0000","last_peered":"2023-12-14T18:24:41.940761+0000","last_clean":"2023-12-14T18:24:41.940761+0000","last_became_active":"2023-12-14T18:24:36.590779+0000","last_became_peered":"2023-12-14T18:24:36.590779+0000","last_unstale":"2023-12-14T18:24:41.940761+0000","last_undegraded":"2023-12-14T18:24:41.940761+0000","last_fullsized":"2023-12-14T18:24:41.940761+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:16:00.885373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940628+0000","last_change":"2023-12-14T18:24:40.604125+0000","last_active":"2023-12-14T18:24:41.940628+0000","last_peered":"2023-12-14T18:24:41.940628+0000","last_clean":"2023-12-14T18:24:41.940628+0000","last_became_active":"2023-12-14T18:24:40.603242+0000","last_became_peered":"2023-12-14T18:24:40.603242+0000","last_unstale":"2023-12-14T18:24:41.940628+0000","last_undegraded":"2023-12-14T18:24:41.940628+0000","last_fullsized":"2023-12-14T18:24:41.940628+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:52:35.630493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845182+0000","last_change":"2023-12-14T18:24:36.177567+0000","last_active":"2023-12-14T18:25:37.845182+0000","last_peered":"2023-12-14T18:25:37.845182+0000","last_clean":"2023-12-14T18:25:37.845182+0000","last_became_active":"2023-12-14T18:24:36.176949+0000","last_became_peered":"2023-12-14T18:24:36.176949+0000","last_unstale":"2023-12-14T18:25:37.845182+0000","last_undegraded":"2023-12-14T18:25:37.845182+0000","last_fullsized":"2023-12-14T18:25:37.845182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:41:12.145991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940032+0000","last_change":"2023-12-14T18:24:36.594224+0000","last_active":"2023-12-14T18:24:41.940032+0000","last_peered":"2023-12-14T18:24:41.940032+0000","last_clean":"2023-12-14T18:24:41.940032+0000","last_became_active":"2023-12-14T18:24:36.593859+0000","last_became_peered":"2023-12-14T18:24:36.593859+0000","last_unstale":"2023-12-14T18:24:41.940032+0000","last_undegraded":"2023-12-14T18:24:41.940032+0000","last_fullsized":"2023-12-14T18:24:41.940032+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:59:04.044943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940173+0000","last_change":"2023-12-14T18:24:40.771814+0000","last_active":"2023-12-14T18:24:41.940173+0000","last_peered":"2023-12-14T18:24:41.940173+0000","last_clean":"2023-12-14T18:24:41.940173+0000","last_became_active":"2023-12-14T18:24:40.771387+0000","last_became_peered":"2023-12-14T18:24:40.771387+0000","last_unstale":"2023-12-14T18:24:41.940173+0000","last_undegraded":"2023-12-14T18:24:41.940173+0000","last_fullsized":"2023-12-14T18:24:41.940173+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:00:22.744059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.846862+0000","last_change":"2023-12-14T18:24:36.588826+0000","last_active":"2023-12-14T18:25:36.846862+0000","last_peered":"2023-12-14T18:25:36.846862+0000","last_clean":"2023-12-14T18:25:36.846862+0000","last_became_active":"2023-12-14T18:24:36.588320+0000","last_became_peered":"2023-12-14T18:24:36.588320+0000","last_unstale":"2023-12-14T18:25:36.846862+0000","last_undegraded":"2023-12-14T18:25:36.846862+0000","last_fullsized":"2023-12-14T18:25:36.846862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:52:29.540309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":61,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:34.832934+0000","last_change":"2023-12-14T18:24:36.173995+0000","last_active":"2023-12-14T18:25:34.832934+0000","last_peered":"2023-12-14T18:25:34.832934+0000","last_clean":"2023-12-14T18:25:34.832934+0000","last_became_active":"2023-12-14T18:24:36.173381+0000","last_became_peered":"2023-12-14T18:24:36.173381+0000","last_unstale":"2023-12-14T18:25:34.832934+0000","last_undegraded":"2023-12-14T18:25:34.832934+0000","last_fullsized":"2023-12-14T18:25:34.832934+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:17:33.452268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844039+0000","last_change":"2023-12-14T18:24:40.605099+0000","last_active":"2023-12-14T18:25:37.844039+0000","last_peered":"2023-12-14T18:25:37.844039+0000","last_clean":"2023-12-14T18:25:37.844039+0000","last_became_active":"2023-12-14T18:24:40.604773+0000","last_became_peered":"2023-12-14T18:24:40.604773+0000","last_unstale":"2023-12-14T18:25:37.844039+0000","last_undegraded":"2023-12-14T18:25:37.844039+0000","last_fullsized":"2023-12-14T18:25:37.844039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:03.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998989+0000","last_change":"2023-12-14T18:24:36.574993+0000","last_active":"2023-12-14T18:24:40.998989+0000","last_peered":"2023-12-14T18:24:40.998989+0000","last_clean":"2023-12-14T18:24:40.998989+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:24:40.998989+0000","last_undegraded":"2023-12-14T18:24:40.998989+0000","last_fullsized":"2023-12-14T18:24:40.998989+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:36:49.384865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.767286+0000","last_change":"2023-12-14T18:24:36.170061+0000","last_active":"2023-12-14T18:25:00.767286+0000","last_peered":"2023-12-14T18:25:00.767286+0000","last_clean":"2023-12-14T18:25:00.767286+0000","last_became_active":"2023-12-14T18:24:36.169652+0000","last_became_peered":"2023-12-14T18:24:36.169652+0000","last_unstale":"2023-12-14T18:25:00.767286+0000","last_undegraded":"2023-12-14T18:25:00.767286+0000","last_fullsized":"2023-12-14T18:25:00.767286+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:35:33.204173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843576+0000","last_change":"2023-12-14T18:23:49.076644+0000","last_active":"2023-12-14T18:25:37.843576+0000","last_peered":"2023-12-14T18:25:37.843576+0000","last_clean":"2023-12-14T18:25:37.843576+0000","last_became_active":"2023-12-14T18:23:49.076263+0000","last_became_peered":"2023-12-14T18:23:49.076263+0000","last_unstale":"2023-12-14T18:25:37.843576+0000","last_undegraded":"2023-12-14T18:25:37.843576+0000","last_fullsized":"2023-12-14T18:25:37.843576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_clean_scrub_stamp":"2023-12-14T18:21:24.715131+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:26:53.561034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847479+0000","last_change":"2023-12-14T18:24:40.772015+0000","last_active":"2023-12-14T18:25:36.847479+0000","last_peered":"2023-12-14T18:25:36.847479+0000","last_clean":"2023-12-14T18:25:36.847479+0000","last_became_active":"2023-12-14T18:24:40.771497+0000","last_became_peered":"2023-12-14T18:24:40.771497+0000","last_unstale":"2023-12-14T18:25:36.847479+0000","last_undegraded":"2023-12-14T18:25:36.847479+0000","last_fullsized":"2023-12-14T18:25:36.847479+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:07:56.111457+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939552+0000","last_change":"2023-12-14T18:24:36.584402+0000","last_active":"2023-12-14T18:24:41.939552+0000","last_peered":"2023-12-14T18:24:41.939552+0000","last_clean":"2023-12-14T18:24:41.939552+0000","last_became_active":"2023-12-14T18:24:36.584177+0000","last_became_peered":"2023-12-14T18:24:36.584177+0000","last_unstale":"2023-12-14T18:24:41.939552+0000","last_undegraded":"2023-12-14T18:24:41.939552+0000","last_fullsized":"2023-12-14T18:24:41.939552+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:38:45.449137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843698+0000","last_change":"2023-12-14T18:24:36.175017+0000","last_active":"2023-12-14T18:25:37.843698+0000","last_peered":"2023-12-14T18:25:37.843698+0000","last_clean":"2023-12-14T18:25:37.843698+0000","last_became_active":"2023-12-14T18:24:36.174639+0000","last_became_peered":"2023-12-14T18:24:36.174639+0000","last_unstale":"2023-12-14T18:25:37.843698+0000","last_undegraded":"2023-12-14T18:25:37.843698+0000","last_fullsized":"2023-12-14T18:25:37.843698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:31:26.105116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.848178+0000","last_change":"2023-12-14T18:25:37.848178+0000","last_active":"2023-12-14T18:25:37.848178+0000","last_peered":"2023-12-14T18:25:37.848178+0000","last_clean":"2023-12-14T18:25:37.848178+0000","last_became_active":"2023-12-14T18:25:37.847717+0000","last_became_peered":"2023-12-14T18:25:37.847717+0000","last_unstale":"2023-12-14T18:25:37.848178+0000","last_undegraded":"2023-12-14T18:25:37.848178+0000","last_fullsized":"2023-12-14T18:25:37.848178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:42:37.509796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.038437+0000","last_change":"2023-12-14T18:24:36.177484+0000","last_active":"2023-12-14T18:25:00.038437+0000","last_peered":"2023-12-14T18:25:00.038437+0000","last_clean":"2023-12-14T18:25:00.038437+0000","last_became_active":"2023-12-14T18:24:36.176592+0000","last_became_peered":"2023-12-14T18:24:36.176592+0000","last_unstale":"2023-12-14T18:25:00.038437+0000","last_undegraded":"2023-12-14T18:25:00.038437+0000","last_fullsized":"2023-12-14T18:25:00.038437+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:21:55.581430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940747+0000","last_change":"2023-12-14T18:24:36.585687+0000","last_active":"2023-12-14T18:24:41.940747+0000","last_peered":"2023-12-14T18:24:41.940747+0000","last_clean":"2023-12-14T18:24:41.940747+0000","last_became_active":"2023-12-14T18:24:36.585360+0000","last_became_peered":"2023-12-14T18:24:36.585360+0000","last_unstale":"2023-12-14T18:24:41.940747+0000","last_undegraded":"2023-12-14T18:24:41.940747+0000","last_fullsized":"2023-12-14T18:24:41.940747+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:30:42.737826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998830+0000","last_change":"2023-12-14T18:24:40.771747+0000","last_active":"2023-12-14T18:24:40.998830+0000","last_peered":"2023-12-14T18:24:40.998830+0000","last_clean":"2023-12-14T18:24:40.998830+0000","last_became_active":"2023-12-14T18:24:40.771310+0000","last_became_peered":"2023-12-14T18:24:40.771310+0000","last_unstale":"2023-12-14T18:24:40.998830+0000","last_undegraded":"2023-12-14T18:24:40.998830+0000","last_fullsized":"2023-12-14T18:24:40.998830+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:55:52.843438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.853179+0000","last_change":"2023-12-14T18:25:37.853179+0000","last_active":"2023-12-14T18:25:37.853179+0000","last_peered":"2023-12-14T18:25:37.853179+0000","last_clean":"2023-12-14T18:25:37.853179+0000","last_became_active":"2023-12-14T18:25:37.848064+0000","last_became_peered":"2023-12-14T18:25:37.848064+0000","last_unstale":"2023-12-14T18:25:37.853179+0000","last_undegraded":"2023-12-14T18:25:37.853179+0000","last_fullsized":"2023-12-14T18:25:37.853179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:47:58.918839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999452+0000","last_change":"2023-12-14T18:24:36.594938+0000","last_active":"2023-12-14T18:24:40.999452+0000","last_peered":"2023-12-14T18:24:40.999452+0000","last_clean":"2023-12-14T18:24:40.999452+0000","last_became_active":"2023-12-14T18:24:36.594573+0000","last_became_peered":"2023-12-14T18:24:36.594573+0000","last_unstale":"2023-12-14T18:24:40.999452+0000","last_undegraded":"2023-12-14T18:24:40.999452+0000","last_fullsized":"2023-12-14T18:24:40.999452+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:18:30.843187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840467+0000","last_change":"2023-12-14T18:24:40.601146+0000","last_active":"2023-12-14T18:25:37.840467+0000","last_peered":"2023-12-14T18:25:37.840467+0000","last_clean":"2023-12-14T18:25:37.840467+0000","last_became_active":"2023-12-14T18:24:40.600745+0000","last_became_peered":"2023-12-14T18:24:40.600745+0000","last_unstale":"2023-12-14T18:25:37.840467+0000","last_undegraded":"2023-12-14T18:25:37.840467+0000","last_fullsized":"2023-12-14T18:25:37.840467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:19:54.844673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":26,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039339+0000","last_change":"2023-12-14T18:24:36.175880+0000","last_active":"2023-12-14T18:25:00.039339+0000","last_peered":"2023-12-14T18:25:00.039339+0000","last_clean":"2023-12-14T18:25:00.039339+0000","last_became_active":"2023-12-14T18:24:36.175514+0000","last_became_peered":"2023-12-14T18:24:36.175514+0000","last_unstale":"2023-12-14T18:25:00.039339+0000","last_undegraded":"2023-12-14T18:25:00.039339+0000","last_fullsized":"2023-12-14T18:25:00.039339+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:00:40.956277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052652+0000","last_change":"2023-12-14T18:24:36.577062+0000","last_active":"2023-12-14T18:25:38.052652+0000","last_peered":"2023-12-14T18:25:38.052652+0000","last_clean":"2023-12-14T18:25:38.052652+0000","last_became_active":"2023-12-14T18:24:36.576733+0000","last_became_peered":"2023-12-14T18:24:36.576733+0000","last_unstale":"2023-12-14T18:25:38.052652+0000","last_undegraded":"2023-12-14T18:25:38.052652+0000","last_fullsized":"2023-12-14T18:25:38.052652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:58.268670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940397+0000","last_change":"2023-12-14T18:24:40.772752+0000","last_active":"2023-12-14T18:24:41.940397+0000","last_peered":"2023-12-14T18:24:41.940397+0000","last_clean":"2023-12-14T18:24:41.940397+0000","last_became_active":"2023-12-14T18:24:40.772168+0000","last_became_peered":"2023-12-14T18:24:40.772168+0000","last_unstale":"2023-12-14T18:24:41.940397+0000","last_undegraded":"2023-12-14T18:24:41.940397+0000","last_fullsized":"2023-12-14T18:24:41.940397+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:02:24.129455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.843859+0000","last_change":"2023-12-14T18:24:36.178885+0000","last_active":"2023-12-14T18:25:36.843859+0000","last_peered":"2023-12-14T18:25:36.843859+0000","last_clean":"2023-12-14T18:25:36.843859+0000","last_became_active":"2023-12-14T18:24:36.178386+0000","last_became_peered":"2023-12-14T18:24:36.178386+0000","last_unstale":"2023-12-14T18:25:36.843859+0000","last_undegraded":"2023-12-14T18:25:36.843859+0000","last_fullsized":"2023-12-14T18:25:36.843859+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:31:44.239186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.095247+0000","last_change":"2023-12-14T18:24:36.590037+0000","last_active":"2023-12-14T18:24:41.095247+0000","last_peered":"2023-12-14T18:24:41.095247+0000","last_clean":"2023-12-14T18:24:41.095247+0000","last_became_active":"2023-12-14T18:24:36.589659+0000","last_became_peered":"2023-12-14T18:24:36.589659+0000","last_unstale":"2023-12-14T18:24:41.095247+0000","last_undegraded":"2023-12-14T18:24:41.095247+0000","last_fullsized":"2023-12-14T18:24:41.095247+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:13:57.949133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.093418+0000","last_change":"2023-12-14T18:24:40.602483+0000","last_active":"2023-12-14T18:24:41.093418+0000","last_peered":"2023-12-14T18:24:41.093418+0000","last_clean":"2023-12-14T18:24:41.093418+0000","last_became_active":"2023-12-14T18:24:40.601989+0000","last_became_peered":"2023-12-14T18:24:40.601989+0000","last_unstale":"2023-12-14T18:24:41.093418+0000","last_undegraded":"2023-12-14T18:24:41.093418+0000","last_fullsized":"2023-12-14T18:24:41.093418+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:09.202695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":25,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.037079+0000","last_change":"2023-12-14T18:24:36.173350+0000","last_active":"2023-12-14T18:25:00.037079+0000","last_peered":"2023-12-14T18:25:00.037079+0000","last_clean":"2023-12-14T18:25:00.037079+0000","last_became_active":"2023-12-14T18:24:36.172840+0000","last_became_peered":"2023-12-14T18:24:36.172840+0000","last_unstale":"2023-12-14T18:25:00.037079+0000","last_undegraded":"2023-12-14T18:25:00.037079+0000","last_fullsized":"2023-12-14T18:25:00.037079+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:17:31.439315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842563+0000","last_change":"2023-12-14T18:24:36.592104+0000","last_active":"2023-12-14T18:25:37.842563+0000","last_peered":"2023-12-14T18:25:37.842563+0000","last_clean":"2023-12-14T18:25:37.842563+0000","last_became_active":"2023-12-14T18:24:36.591568+0000","last_became_peered":"2023-12-14T18:24:36.591568+0000","last_unstale":"2023-12-14T18:25:37.842563+0000","last_undegraded":"2023-12-14T18:25:37.842563+0000","last_fullsized":"2023-12-14T18:25:37.842563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:38:19.772980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":14,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940589+0000","last_change":"2023-12-14T18:24:40.596043+0000","last_active":"2023-12-14T18:24:41.940589+0000","last_peered":"2023-12-14T18:24:41.940589+0000","last_clean":"2023-12-14T18:24:41.940589+0000","last_became_active":"2023-12-14T18:24:40.595520+0000","last_became_peered":"2023-12-14T18:24:40.595520+0000","last_unstale":"2023-12-14T18:24:41.940589+0000","last_undegraded":"2023-12-14T18:24:41.940589+0000","last_fullsized":"2023-12-14T18:24:41.940589+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:45:44.615227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.856268+0000","last_change":"2023-12-14T18:25:37.855817+0000","last_active":"2023-12-14T18:25:37.856268+0000","last_peered":"2023-12-14T18:25:37.856268+0000","last_clean":"2023-12-14T18:25:37.856268+0000","last_became_active":"2023-12-14T18:25:37.849192+0000","last_became_peered":"2023-12-14T18:25:37.849192+0000","last_unstale":"2023-12-14T18:25:37.856268+0000","last_undegraded":"2023-12-14T18:25:37.856268+0000","last_fullsized":"2023-12-14T18:25:37.856268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:23:11.825835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999206+0000","last_change":"2023-12-14T18:24:36.587777+0000","last_active":"2023-12-14T18:24:40.999206+0000","last_peered":"2023-12-14T18:24:40.999206+0000","last_clean":"2023-12-14T18:24:40.999206+0000","last_became_active":"2023-12-14T18:24:36.587317+0000","last_became_peered":"2023-12-14T18:24:36.587317+0000","last_unstale":"2023-12-14T18:24:40.999206+0000","last_undegraded":"2023-12-14T18:24:40.999206+0000","last_fullsized":"2023-12-14T18:24:40.999206+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:59:49.931215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":13,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940400+0000","last_change":"2023-12-14T18:24:40.771913+0000","last_active":"2023-12-14T18:24:41.940400+0000","last_peered":"2023-12-14T18:24:41.940400+0000","last_clean":"2023-12-14T18:24:41.940400+0000","last_became_active":"2023-12-14T18:24:40.771425+0000","last_became_peered":"2023-12-14T18:24:40.771425+0000","last_unstale":"2023-12-14T18:24:41.940400+0000","last_undegraded":"2023-12-14T18:24:41.940400+0000","last_fullsized":"2023-12-14T18:24:41.940400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:17:43.378002+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":27,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:25:00.039457+0000","last_change":"2023-12-14T18:24:36.176801+0000","last_active":"2023-12-14T18:25:00.039457+0000","last_peered":"2023-12-14T18:25:00.039457+0000","last_clean":"2023-12-14T18:25:00.039457+0000","last_became_active":"2023-12-14T18:24:36.176001+0000","last_became_peered":"2023-12-14T18:24:36.176001+0000","last_unstale":"2023-12-14T18:25:00.039457+0000","last_undegraded":"2023-12-14T18:25:00.039457+0000","last_fullsized":"2023-12-14T18:25:00.039457+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:16:31.583481+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845322+0000","last_change":"2023-12-14T18:24:36.592898+0000","last_active":"2023-12-14T18:25:37.845322+0000","last_peered":"2023-12-14T18:25:37.845322+0000","last_clean":"2023-12-14T18:25:37.845322+0000","last_became_active":"2023-12-14T18:24:36.586650+0000","last_became_peered":"2023-12-14T18:24:36.586650+0000","last_unstale":"2023-12-14T18:25:37.845322+0000","last_undegraded":"2023-12-14T18:25:37.845322+0000","last_fullsized":"2023-12-14T18:25:37.845322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:35:59.247019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.845122+0000","last_change":"2023-12-14T18:24:40.599181+0000","last_active":"2023-12-14T18:25:36.845122+0000","last_peered":"2023-12-14T18:25:36.845122+0000","last_clean":"2023-12-14T18:25:36.845122+0000","last_became_active":"2023-12-14T18:24:40.598785+0000","last_became_peered":"2023-12-14T18:24:40.598785+0000","last_unstale":"2023-12-14T18:25:36.845122+0000","last_undegraded":"2023-12-14T18:25:36.845122+0000","last_fullsized":"2023-12-14T18:25:36.845122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:01:59.215376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841700+0000","last_change":"2023-12-14T18:24:36.173644+0000","last_active":"2023-12-14T18:25:37.841700+0000","last_peered":"2023-12-14T18:25:37.841700+0000","last_clean":"2023-12-14T18:25:37.841700+0000","last_became_active":"2023-12-14T18:24:36.173059+0000","last_became_peered":"2023-12-14T18:24:36.173059+0000","last_unstale":"2023-12-14T18:25:37.841700+0000","last_undegraded":"2023-12-14T18:25:37.841700+0000","last_fullsized":"2023-12-14T18:25:37.841700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:16:27.909550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844525+0000","last_change":"2023-12-14T18:24:36.588761+0000","last_active":"2023-12-14T18:25:37.844525+0000","last_peered":"2023-12-14T18:25:37.844525+0000","last_clean":"2023-12-14T18:25:37.844525+0000","last_became_active":"2023-12-14T18:24:36.586021+0000","last_became_peered":"2023-12-14T18:24:36.586021+0000","last_unstale":"2023-12-14T18:25:37.844525+0000","last_undegraded":"2023-12-14T18:25:37.844525+0000","last_fullsized":"2023-12-14T18:25:37.844525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:45:41.729490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052349+0000","last_change":"2023-12-14T18:24:40.607286+0000","last_active":"2023-12-14T18:25:38.052349+0000","last_peered":"2023-12-14T18:25:38.052349+0000","last_clean":"2023-12-14T18:25:38.052349+0000","last_became_active":"2023-12-14T18:24:40.606925+0000","last_became_peered":"2023-12-14T18:24:40.606925+0000","last_unstale":"2023-12-14T18:25:38.052349+0000","last_undegraded":"2023-12-14T18:25:38.052349+0000","last_fullsized":"2023-12-14T18:25:38.052349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:26:27.152905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":26,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.847782+0000","last_change":"2023-12-14T18:24:36.174467+0000","last_active":"2023-12-14T18:25:36.847782+0000","last_peered":"2023-12-14T18:25:36.847782+0000","last_clean":"2023-12-14T18:25:36.847782+0000","last_became_active":"2023-12-14T18:24:36.173920+0000","last_became_peered":"2023-12-14T18:24:36.173920+0000","last_unstale":"2023-12-14T18:25:36.847782+0000","last_undegraded":"2023-12-14T18:25:36.847782+0000","last_fullsized":"2023-12-14T18:25:36.847782+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:49:56.456390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846258+0000","last_change":"2023-12-14T18:24:36.597537+0000","last_active":"2023-12-14T18:25:37.846258+0000","last_peered":"2023-12-14T18:25:37.846258+0000","last_clean":"2023-12-14T18:25:37.846258+0000","last_became_active":"2023-12-14T18:24:36.594599+0000","last_became_peered":"2023-12-14T18:24:36.594599+0000","last_unstale":"2023-12-14T18:25:37.846258+0000","last_undegraded":"2023-12-14T18:25:37.846258+0000","last_fullsized":"2023-12-14T18:25:37.846258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:00:18.245087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":15,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.846101+0000","last_change":"2023-12-14T18:24:40.597990+0000","last_active":"2023-12-14T18:25:36.846101+0000","last_peered":"2023-12-14T18:25:36.846101+0000","last_clean":"2023-12-14T18:25:36.846101+0000","last_became_active":"2023-12-14T18:24:40.597568+0000","last_became_peered":"2023-12-14T18:24:40.597568+0000","last_unstale":"2023-12-14T18:25:36.846101+0000","last_undegraded":"2023-12-14T18:25:36.846101+0000","last_fullsized":"2023-12-14T18:25:36.846101+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:21:01.498662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":24,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940025+0000","last_change":"2023-12-14T18:24:36.173141+0000","last_active":"2023-12-14T18:24:41.940025+0000","last_peered":"2023-12-14T18:24:41.940025+0000","last_clean":"2023-12-14T18:24:41.940025+0000","last_became_active":"2023-12-14T18:24:36.172811+0000","last_became_peered":"2023-12-14T18:24:36.172811+0000","last_unstale":"2023-12-14T18:24:41.940025+0000","last_undegraded":"2023-12-14T18:24:41.940025+0000","last_fullsized":"2023-12-14T18:24:41.940025+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:01:41.375107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844789+0000","last_change":"2023-12-14T18:24:36.587333+0000","last_active":"2023-12-14T18:25:37.844789+0000","last_peered":"2023-12-14T18:25:37.844789+0000","last_clean":"2023-12-14T18:25:37.844789+0000","last_became_active":"2023-12-14T18:24:36.586914+0000","last_became_peered":"2023-12-14T18:24:36.586914+0000","last_unstale":"2023-12-14T18:25:37.844789+0000","last_undegraded":"2023-12-14T18:25:37.844789+0000","last_fullsized":"2023-12-14T18:25:37.844789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:48:31.035981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842775+0000","last_change":"2023-12-14T18:24:36.591896+0000","last_active":"2023-12-14T18:25:37.842775+0000","last_peered":"2023-12-14T18:25:37.842775+0000","last_clean":"2023-12-14T18:25:37.842775+0000","last_became_active":"2023-12-14T18:24:36.591097+0000","last_became_peered":"2023-12-14T18:24:36.591097+0000","last_unstale":"2023-12-14T18:25:37.842775+0000","last_undegraded":"2023-12-14T18:25:37.842775+0000","last_fullsized":"2023-12-14T18:25:37.842775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:22:08.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841677+0000","last_change":"2023-12-14T18:24:36.581688+0000","last_active":"2023-12-14T18:25:37.841677+0000","last_peered":"2023-12-14T18:25:37.841677+0000","last_clean":"2023-12-14T18:25:37.841677+0000","last_became_active":"2023-12-14T18:24:36.580660+0000","last_became_peered":"2023-12-14T18:24:36.580660+0000","last_unstale":"2023-12-14T18:25:37.841677+0000","last_undegraded":"2023-12-14T18:25:37.841677+0000","last_fullsized":"2023-12-14T18:25:37.841677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:11:30.240797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840500+0000","last_change":"2023-12-14T18:24:36.583025+0000","last_active":"2023-12-14T18:25:37.840500+0000","last_peered":"2023-12-14T18:25:37.840500+0000","last_clean":"2023-12-14T18:25:37.840500+0000","last_became_active":"2023-12-14T18:24:36.582510+0000","last_became_peered":"2023-12-14T18:24:36.582510+0000","last_unstale":"2023-12-14T18:25:37.840500+0000","last_undegraded":"2023-12-14T18:25:37.840500+0000","last_fullsized":"2023-12-14T18:25:37.840500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:34:09.434901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052493+0000","last_change":"2023-12-14T18:24:36.579383+0000","last_active":"2023-12-14T18:25:38.052493+0000","last_peered":"2023-12-14T18:25:38.052493+0000","last_clean":"2023-12-14T18:25:38.052493+0000","last_became_active":"2023-12-14T18:24:36.579037+0000","last_became_peered":"2023-12-14T18:24:36.579037+0000","last_unstale":"2023-12-14T18:25:38.052493+0000","last_undegraded":"2023-12-14T18:25:38.052493+0000","last_fullsized":"2023-12-14T18:25:38.052493+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:20:41.833469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.940749+0000","last_change":"2023-12-14T18:24:36.582940+0000","last_active":"2023-12-14T18:24:41.940749+0000","last_peered":"2023-12-14T18:24:41.940749+0000","last_clean":"2023-12-14T18:24:41.940749+0000","last_became_active":"2023-12-14T18:24:36.582645+0000","last_became_peered":"2023-12-14T18:24:36.582645+0000","last_unstale":"2023-12-14T18:24:41.940749+0000","last_undegraded":"2023-12-14T18:24:41.940749+0000","last_fullsized":"2023-12-14T18:24:41.940749+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:43:46.761797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843497+0000","last_change":"2023-12-14T18:24:36.578613+0000","last_active":"2023-12-14T18:25:37.843497+0000","last_peered":"2023-12-14T18:25:37.843497+0000","last_clean":"2023-12-14T18:25:37.843497+0000","last_became_active":"2023-12-14T18:24:36.578287+0000","last_became_peered":"2023-12-14T18:24:36.578287+0000","last_unstale":"2023-12-14T18:25:37.843497+0000","last_undegraded":"2023-12-14T18:25:37.843497+0000","last_fullsized":"2023-12-14T18:25:37.843497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:09:07.746792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841403+0000","last_change":"2023-12-14T18:24:36.575078+0000","last_active":"2023-12-14T18:25:37.841403+0000","last_peered":"2023-12-14T18:25:37.841403+0000","last_clean":"2023-12-14T18:25:37.841403+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:25:37.841403+0000","last_undegraded":"2023-12-14T18:25:37.841403+0000","last_fullsized":"2023-12-14T18:25:37.841403+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:19:17.093417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.999571+0000","last_change":"2023-12-14T18:24:36.572545+0000","last_active":"2023-12-14T18:24:40.999571+0000","last_peered":"2023-12-14T18:24:40.999571+0000","last_clean":"2023-12-14T18:24:40.999571+0000","last_became_active":"2023-12-14T18:24:36.572235+0000","last_became_peered":"2023-12-14T18:24:36.572235+0000","last_unstale":"2023-12-14T18:24:40.999571+0000","last_undegraded":"2023-12-14T18:24:40.999571+0000","last_fullsized":"2023-12-14T18:24:40.999571+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:25:56.081431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.846251+0000","last_change":"2023-12-14T18:24:36.588278+0000","last_active":"2023-12-14T18:25:36.846251+0000","last_peered":"2023-12-14T18:25:36.846251+0000","last_clean":"2023-12-14T18:25:36.846251+0000","last_became_active":"2023-12-14T18:24:36.587605+0000","last_became_peered":"2023-12-14T18:24:36.587605+0000","last_unstale":"2023-12-14T18:25:36.846251+0000","last_undegraded":"2023-12-14T18:25:36.846251+0000","last_fullsized":"2023-12-14T18:25:36.846251+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:52.787724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:40.998910+0000","last_change":"2023-12-14T18:24:36.575325+0000","last_active":"2023-12-14T18:24:40.998910+0000","last_peered":"2023-12-14T18:24:40.998910+0000","last_clean":"2023-12-14T18:24:40.998910+0000","last_became_active":"2023-12-14T18:24:36.575016+0000","last_became_peered":"2023-12-14T18:24:36.575016+0000","last_unstale":"2023-12-14T18:24:40.998910+0000","last_undegraded":"2023-12-14T18:24:40.998910+0000","last_fullsized":"2023-12-14T18:24:40.998910+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:35:26.394701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":22,"reported_epoch":85,"state":"active+clean","last_fresh":"2023-12-14T18:25:36.845337+0000","last_change":"2023-12-14T18:24:36.585348+0000","last_active":"2023-12-14T18:25:36.845337+0000","last_peered":"2023-12-14T18:25:36.845337+0000","last_clean":"2023-12-14T18:25:36.845337+0000","last_became_active":"2023-12-14T18:24:36.584995+0000","last_became_peered":"2023-12-14T18:24:36.584995+0000","last_unstale":"2023-12-14T18:25:36.845337+0000","last_undegraded":"2023-12-14T18:25:36.845337+0000","last_fullsized":"2023-12-14T18:25:36.845337+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:48:21.622579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":20,"reported_epoch":84,"state":"active+clean","last_fresh":"2023-12-14T18:24:41.939979+0000","last_change":"2023-12-14T18:24:36.583571+0000","last_active":"2023-12-14T18:24:41.939979+0000","last_peered":"2023-12-14T18:24:41.939979+0000","last_clean":"2023-12-14T18:24:41.939979+0000","last_became_active":"2023-12-14T18:24:36.583266+0000","last_became_peered":"2023-12-14T18:24:36.583266+0000","last_unstale":"2023-12-14T18:24:41.939979+0000","last_undegraded":"2023-12-14T18:24:41.939979+0000","last_fullsized":"2023-12-14T18:24:41.939979+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:09:22.877080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843165+0000","last_change":"2023-12-14T18:24:36.579754+0000","last_active":"2023-12-14T18:25:37.843165+0000","last_peered":"2023-12-14T18:25:37.843165+0000","last_clean":"2023-12-14T18:25:37.843165+0000","last_became_active":"2023-12-14T18:24:36.579337+0000","last_became_peered":"2023-12-14T18:24:36.579337+0000","last_unstale":"2023-12-14T18:25:37.843165+0000","last_undegraded":"2023-12-14T18:25:37.843165+0000","last_fullsized":"2023-12-14T18:25:37.843165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:17:05.199500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":28672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212533,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64740,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":236,"num_read_kb":189,"num_write":153,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":3,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":409600,"data_stored":193256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547223,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6112,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739040,"statfs":{"total":95995035648,"available":95988776960,"internally_reserved":0,"allocated":942080,"data_stored":477716,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.313},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.433},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.497}]},{"osd":7,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.53300000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.57599999999999996}]}]},{"osd":10,"up_from":67,"seq":287762808858,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":1032,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":1056768,"data_stored":1106365,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.6690000000000005,"5min":8.6690000000000005,"15min":8.6690000000000005},"last":0.26700000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.36299999999999999},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":9.5459999999999994,"5min":9.5459999999999994,"15min":9.5459999999999994},"last":0.33300000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":9.4380000000000006,"5min":9.4380000000000006,"15min":9.4380000000000006},"last":0.498},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.46100000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.9470000000000001,"5min":5.9470000000000001,"15min":5.9470000000000001},"last":0.379},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":6.1470000000000002,"5min":6.1470000000000002,"15min":6.1470000000000002},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.40000000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":9.4830000000000005,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":0.439},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.53400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.48399999999999999}]},{"osd":9,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.54100000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":6.0309999999999997,"5min":6.0309999999999997,"15min":6.0309999999999997},"last":0.52400000000000002}]}]},{"osd":9,"up_from":62,"seq":266287972381,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6248,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5312,"kb_avail":93738904,"statfs":{"total":95995035648,"available":95988637696,"internally_reserved":0,"allocated":950272,"data_stored":472482,"data_compressed":14585,"data_compressed_allocated":20480,"data_compressed_original":40960,"omap_allocated":0,"internal_metadata":5439488},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42199999999999999,"15min":0.42999999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53500000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.495},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.27200000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.73999999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.59399999999999997}]},{"osd":1,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60999999999999999,"15min":0.62},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.83399999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.755}]},{"osd":2,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61199999999999999,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.65100000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.47199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59199999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.879,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.64000000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72999999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.66300000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.45900000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80700000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.749},"min":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84799999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.69199999999999995}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.81699999999999995,"5min":1.214,"15min":1.214},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.47199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79600000000000004,"5min":1.071,"15min":1.071},"last":0.70099999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.747},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.73699999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76400000000000001}]}]},{"osd":8,"up_from":55,"seq":236223201313,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6356,"kb_used_data":908,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738796,"statfs":{"total":95995035648,"available":95988527104,"internally_reserved":0,"allocated":929792,"data_stored":458465,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.51400000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.501},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.372},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.39100000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.65800000000000003}]},{"osd":4,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.53700000000000003}]},{"osd":5,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.41299999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.69899999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.47599999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.628},{"interface":"front","average":{"1min":0.754,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.71699999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60199999999999998}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58299999999999996}]}]},{"osd":7,"up_from":48,"seq":206158430244,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6544,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738608,"statfs":{"total":95995035648,"available":95988334592,"internally_reserved":0,"allocated":991232,"data_stored":1063787,"data_compressed":50280,"data_compressed_allocated":65536,"data_compressed_original":630784,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.434}]},{"osd":1,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.38700000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.47899999999999998}]},{"osd":3,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.498}]},{"osd":4,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.53600000000000003,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.7830000000000004,"5min":5.7830000000000004,"15min":5.7830000000000004},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.29499999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.55500000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.52000000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77500000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.39900000000000002}]},{"osd":8,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.58999999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82699999999999996,"5min":0.879,"15min":0.879},"last":0.68000000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.56399999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.53100000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.50600000000000001}]},{"osd":11,"last update":"Thu Dec 14 18:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.68899999999999995}]}]},{"osd":6,"up_from":43,"seq":184683593766,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6340,"kb_used_data":892,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738812,"statfs":{"total":95995035648,"available":95988543488,"internally_reserved":0,"allocated":913408,"data_stored":454052,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.40699999999999997,"5min":0.38800000000000001,"15min":0.38500000000000001},"min":{"1min":0.23400000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40600000000000003,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.253,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"last":0.30299999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.52800000000000002},"min":{"1min":0.34300000000000003,"5min":0.253,"15min":0.253},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.53200000000000003,"15min":0.53000000000000003},"min":{"1min":0.40500000000000003,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.70399999999999996,"5min":0.77100000000000002,"15min":0.77100000000000002},"last":0.48399999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.55800000000000005,"15min":0.56200000000000006},"min":{"1min":0.38800000000000001,"5min":0.308,"15min":0.308},"max":{"1min":0.745,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.55500000000000005,"15min":0.55300000000000005},"min":{"1min":0.39200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.51400000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46000000000000002,"15min":0.46200000000000002},"min":{"1min":0.28100000000000003,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.70899999999999996,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"min":{"1min":0.29499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.46400000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.78400000000000003,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.58599999999999997,"15min":0.58799999999999997},"min":{"1min":0.39600000000000002,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.79100000000000004,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.53000000000000003}]},{"osd":5,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.625,"15min":0.629},"min":{"1min":0.439,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.748,"5min":0.90700000000000003,"15min":0.90700000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.629,"15min":0.63400000000000001},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.76700000000000002,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54400000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.624,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.56499999999999995}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.65200000000000002,"15min":0.65700000000000003},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79200000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.501},{"interface":"front","average":{"1min":0.627,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79800000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.47199999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.60199999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.59499999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.623}]}]},{"osd":1,"up_from":13,"seq":55834574906,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7364,"kb_used_data":956,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737788,"statfs":{"total":95995035648,"available":95987494912,"internally_reserved":0,"allocated":978944,"data_stored":514798,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.46200000000000002},"min":{"1min":0.36099999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.47299999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.42999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.41299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.42799999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47799999999999998,"15min":0.46500000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.48499999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.316,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.40300000000000002},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77600000000000002,"5min":1.542,"15min":1.542},"last":0.34799999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53800000000000003,"15min":0.51900000000000002},"min":{"1min":0.442,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.501}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.56399999999999995,"15min":0.56100000000000005},"min":{"1min":0.44600000000000001,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.54200000000000004,"15min":0.54100000000000004},"min":{"1min":0.39200000000000002,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.76100000000000001,"5min":0.78600000000000003,"15min":0.78600000000000003},"last":0.67000000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83299999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.63600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.501,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.57899999999999996}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52900000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.72299999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.65100000000000002}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.749}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7220,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737932,"statfs":{"total":95995035648,"available":95987642368,"internally_reserved":0,"allocated":962560,"data_stored":474516,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.52500000000000002,"15min":0.497},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54000000000000004,"15min":0.5},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.27300000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.378},{"interface":"front","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35499999999999998},"min":{"1min":0.26700000000000002,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.38,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51200000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71799999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.51300000000000001,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.40400000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.66200000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.63300000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.63900000000000001,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":7.2290000000000001,"5min":7.2290000000000001,"15min":7.2290000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.67300000000000004,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.37,"15min":0.37},"max":{"1min":7.3390000000000004,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.64100000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.623}]},{"osd":11,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65200000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411382,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7532,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737620,"statfs":{"total":95995035648,"available":95987322880,"internally_reserved":0,"allocated":1019904,"data_stored":534602,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47599999999999998,"15min":0.442},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.45500000000000002},"min":{"1min":0.377,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.54600000000000004,"15min":0.48699999999999999},"min":{"1min":0.51100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.55900000000000005,"15min":0.501},"min":{"1min":0.45600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.46600000000000003},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76100000000000001,"5min":1.179,"15min":1.179},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Dec 14 18:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.748}]},{"osd":7,"last update":"Thu Dec 14 18:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53000000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73099999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.625},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.70599999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71999999999999997}]},{"osd":11,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.80900000000000005}]}]},{"osd":3,"up_from":24,"seq":103079215154,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":925696,"data_stored":454246,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.32800000000000001},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.503},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.248,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.48699999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.497,"15min":0.47499999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.626}]},{"osd":2,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.54800000000000004}]},{"osd":4,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":1.214,"15min":1.214},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.45300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.86399999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64100000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.65700000000000003},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.66400000000000003}]},{"osd":6,"last update":"Thu Dec 14 18:24:38 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.497,"15min":0.499},"min":{"1min":0.33000000000000002,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.71499999999999997,"5min":0.755,"15min":0.755},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.53000000000000003,"15min":0.53600000000000003},"min":{"1min":0.28199999999999997,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.70099999999999996,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.64500000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:24:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.628},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.85099999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79900000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.63700000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.63700000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.83199999999999996}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72999999999999998}]}]},{"osd":4,"up_from":31,"seq":133143986223,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6480,"kb_used_data":904,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738672,"statfs":{"total":95995035648,"available":95988400128,"internally_reserved":0,"allocated":925696,"data_stored":458654,"data_compressed":11634,"data_compressed_allocated":16384,"data_compressed_original":32768,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48799999999999999,"15min":0.48099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.30499999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.622},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.39900000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.48399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.50700000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.45000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.76500000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.44400000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:24:39 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.55900000000000005,"15min":0.55100000000000005},"min":{"1min":0.435,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.58699999999999997,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.83099999999999996,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.55400000000000005}]},{"osd":7,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.36899999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.503,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.59799999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.51900000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.84499999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.64600000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.61199999999999999}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":5,"up_from":36,"seq":154618822699,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6608,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":5632,"kb_avail":93738544,"statfs":{"total":95995035648,"available":95988269056,"internally_reserved":0,"allocated":991232,"data_stored":1050244,"data_compressed":47329,"data_compressed_allocated":61440,"data_compressed_original":622592,"omap_allocated":0,"internal_metadata":5767168},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.626,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.34300000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.501,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.56899999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.254,"15min":0.254},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.32800000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.628,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.32000000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.68000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.498},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58699999999999997}]},{"osd":6,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.61299999999999999,"15min":0.61799999999999999},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.80400000000000005,"5min":0.997,"15min":0.997},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90000000000000002,"5min":0.90000000000000002,"15min":0.90000000000000002},"last":0.42899999999999999}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.67900000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82299999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.41399999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.58599999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.61799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.50600000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69499999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":596,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:25:41.379 INFO:tasks.cephadm.ceph_manager.ceph:PG 2.1b is not active+clean 2023-12-14T18:25:41.379 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '2.1b', 'version': "84'6", 'reported_seq': 69, 'reported_epoch': 84, 'state': 'peering', 'last_fresh': '2023-12-14T18:25:36.813236+0000', 'last_change': '2023-12-14T18:25:36.845980+0000', 'last_active': '2023-12-14T18:25:36.845980+0000', 'last_peered': '2023-12-14T18:25:36.813236+0000', 'last_clean': '2023-12-14T18:25:36.813236+0000', 'last_became_active': '2023-12-14T18:24:36.175965+0000', 'last_became_peered': '2023-12-14T18:24:36.175965+0000', 'last_unstale': '2023-12-14T18:25:36.813236+0000', 'last_undegraded': '2023-12-14T18:25:36.813236+0000', 'last_fullsized': '2023-12-14T18:25:36.813236+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 77, 'last_epoch_clean': 78, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-14T18:24:35.142678+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-14T18:24:35.142678+0000', 'last_clean_scrub_stamp': '2023-12-14T18:24:35.142678+0000', 'objects_scrubbed': 0, 'log_size': 6, 'log_dups_size': 0, 'ondisk_log_size': 6, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'no scrub is scheduled', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 964, 'num_objects': 2, 'num_object_clones': 0, 'num_object_copies': 6, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 2, 'num_whiteouts': 0, 'num_read': 35, 'num_read_kb': 3, 'num_write': 3, 'num_write_kb': 3, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 4, 7], 'acting': [5, 4, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [2, 4, 7], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-14T18:25:41.379 INFO:tasks.cephadm.ceph_manager.ceph:PG 3.10 is not active+clean 2023-12-14T18:25:41.379 INFO:tasks.cephadm.ceph_manager.ceph:{'pgid': '3.10', 'version': "0'0", 'reported_seq': 24, 'reported_epoch': 86, 'state': 'activating', 'last_fresh': '2023-12-14T18:25:37.844035+0000', 'last_change': '2023-12-14T18:25:37.844035+0000', 'last_active': '2023-12-14T18:25:36.841095+0000', 'last_peered': '2023-12-14T18:24:41.939941+0000', 'last_clean': '2023-12-14T18:24:41.939941+0000', 'last_became_active': '2023-12-14T18:24:36.596900+0000', 'last_became_peered': '2023-12-14T18:24:36.596900+0000', 'last_unstale': '2023-12-14T18:25:37.844035+0000', 'last_undegraded': '2023-12-14T18:25:37.844035+0000', 'last_fullsized': '2023-12-14T18:25:37.844035+0000', 'mapping_epoch': 85, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 78, 'last_epoch_clean': 79, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-14T18:24:36.150498+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-14T18:24:36.150498+0000', 'last_clean_scrub_stamp': '2023-12-14T18:24:36.150498+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-15T22:19:00.666432+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [7, 9, 10], 'acting': [7, 9, 10], 'avail_no_missing': ['7', '5', '9', '10'], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 7, 'acting_primary': 7, 'purged_snaps': []} 2023-12-14T18:25:41.380 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph status --format=json 2023-12-14T18:25:41.693 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:42.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:41 smithi093 ceph-mon[42070]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:42.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:41 smithi160 ceph-mon[43840]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:42.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:41 smithi097 ceph-mon[43486]: from='client.15105 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:42.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2998172369 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ffe10 msgr2=0x7f19dc103de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:42.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2998172369 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ffe10 0x7f19dc103de0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f19c40099c0 tx=0x7f19c4009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2998172369 shutdown_connections 2023-12-14T18:25:42.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2998172369 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19dc104510 0x7f19dc10c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2998172369 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19dc0ff470 0x7f19dc0ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2998172369 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ffe10 0x7f19dc103de0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2998172369 >> 172.21.15.93:0/2998172369 conn(0x7f19dc0770d0 msgr2=0x7f19dc0774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:42.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2998172369 shutdown_connections 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2998172369 wait complete. 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.707+0000 7f19e1bb2700 1 Processor -- start 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19e1bb2700 1 -- start start 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19e1bb2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ff470 0x7f19dc195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19e1bb2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19dc0ffe10 0x7f19dc1958a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19e1bb2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19dc104510 0x7f19dc199d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19e1bb2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc19a320 con 0x7f19dc0ff470 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19e1bb2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc19a490 con 0x7f19dc0ffe10 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19e1bb2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19dc19a650 con 0x7f19dc104510 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19db7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ff470 0x7f19dc195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19db7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ff470 0x7f19dc195360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35528/0 (socket says 172.21.15.93:35528) 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19db7fe700 1 -- 172.21.15.93:0/2394807993 learned_addr learned my addr 172.21.15.93:0/2394807993 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19dbfff700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19dc104510 0x7f19dc199d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:42.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19daffd700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19dc0ffe10 0x7f19dc1958a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19db7fe700 1 -- 172.21.15.93:0/2394807993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19dc104510 msgr2=0x7f19dc199d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19db7fe700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19dc104510 0x7f19dc199d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19db7fe700 1 -- 172.21.15.93:0/2394807993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19dc0ffe10 msgr2=0x7f19dc1958a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19db7fe700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19dc0ffe10 0x7f19dc1958a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19db7fe700 1 -- 172.21.15.93:0/2394807993 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19c40095f0 con 0x7f19dc0ff470 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.708+0000 7f19db7fe700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ff470 0x7f19dc195360 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f19d400ef10 tx=0x7f19d400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.709+0000 7f19d8ff9700 1 -- 172.21.15.93:0/2394807993 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d400ca70 con 0x7f19dc0ff470 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.709+0000 7f19d8ff9700 1 -- 172.21.15.93:0/2394807993 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19d400f040 con 0x7f19dc0ff470 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.709+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19dc10f240 con 0x7f19dc0ff470 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.709+0000 7f19d8ff9700 1 -- 172.21.15.93:0/2394807993 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19d4014480 con 0x7f19dc0ff470 2023-12-14T18:25:42.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.709+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19dc10f0b0 con 0x7f19dc0ff470 2023-12-14T18:25:42.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.710+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19dc10f3d0 con 0x7f19dc0ff470 2023-12-14T18:25:42.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.711+0000 7f19d8ff9700 1 -- 172.21.15.93:0/2394807993 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f19d4007500 con 0x7f19dc0ff470 2023-12-14T18:25:42.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.711+0000 7f19d8ff9700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19c80a11e0 0x7f19c80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:42.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.711+0000 7f19daffd700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19c80a11e0 0x7f19c80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:42.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.712+0000 7f19d8ff9700 1 -- 172.21.15.93:0/2394807993 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f19d40cd750 con 0x7f19dc0ff470 2023-12-14T18:25:42.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.712+0000 7f19daffd700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19c80a11e0 0x7f19c80a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009fd0 tx=0x7f19c401a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:42.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.715+0000 7f19d8ff9700 1 -- 172.21.15.93:0/2394807993 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f19d4096a30 con 0x7f19dc0ff470 2023-12-14T18:25:42.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.985+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f19dc066ea0 con 0x7f19dc0ff470 2023-12-14T18:25:42.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.986+0000 7f19d8ff9700 1 -- 172.21.15.93:0/2394807993 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+2449 (secure 0 0 0) 0x7f19d409a050 con 0x7f19dc0ff470 2023-12-14T18:25:42.988 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:42.988 INFO:teuthology.orchestra.run.smithi093.stdout:{"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":14,"quorum":[0,1,2],"quorum_names":["a","c","b"],"quorum_age":324,"monmap":{"epoch":3,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":86,"num_osds":12,"num_up_osds":12,"osd_up_since":1702578243,"num_in_osds":12,"osd_in_since":1702578226,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":127},{"state_name":"peering","count":1},{"state_name":"activating","count":1}],"num_pgs":129,"num_pools":4,"num_objects":64,"data_bytes":655354,"bytes_used":82165760,"bytes_avail":1151858262016,"bytes_total":1151940427776,"inactive_pgs_ratio":0.015503875911235809,"recovering_objects_per_sec":0,"recovering_bytes_per_sec":15,"recovering_keys_per_sec":0,"num_objects_recovered":4,"num_bytes_recovered":123,"num_keys_recovered":7,"read_bytes_sec":4094,"write_bytes_sec":255,"read_op_per_sec":6,"write_op_per_sec":0},"fsmap":{"epoch":24,"id":1,"up":3,"in":3,"max":3,"by_rank":[{"filesystem_id":1,"rank":2,"name":"f","status":"up:standby-replay","gid":24415},{"filesystem_id":1,"rank":2,"name":"e","status":"up:active","gid":24422},{"filesystem_id":1,"rank":1,"name":"c","status":"up:standby-replay","gid":24424},{"filesystem_id":1,"rank":0,"name":"h","status":"up:standby-replay","gid":24440},{"filesystem_id":1,"rank":1,"name":"i","status":"up:active","gid":24442},{"filesystem_id":1,"rank":0,"name":"b","status":"up:active","gid":24461}],"up:standby":6},"mgrmap":{"available":true,"num_standbys":2,"modules":["cephadm","dashboard","iostat","nfs","restful"],"services":{"dashboard":"https://172.21.15.93:8443/"}},"servicemap":{"epoch":22,"modified":"2023-12-14T18:25:41.962207+0000","services":{"mds":{"daemons":{"summary":"","c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"e":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"f":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"h":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"i":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-14T18:25:42.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19c80a11e0 msgr2=0x7f19c80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:42.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19c80a11e0 0x7f19c80a3690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f19c4009fd0 tx=0x7f19c401a040 comp rx=0 tx=0).stop 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ff470 msgr2=0x7f19dc195360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ff470 0x7f19dc195360 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f19d400ef10 tx=0x7f19d400c3a0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 shutdown_connections 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19c80a11e0 0x7f19c80a3690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19dc0ff470 0x7f19dc195360 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19dc104510 0x7f19dc199d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 --2- 172.21.15.93:0/2394807993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19dc0ffe10 0x7f19dc1958a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 >> 172.21.15.93:0/2394807993 conn(0x7f19dc0770d0 msgr2=0x7f19dc0fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 shutdown_connections 2023-12-14T18:25:42.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:42.988+0000 7f19e1bb2700 1 -- 172.21.15.93:0/2394807993 wait complete. 2023-12-14T18:25:43.142 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:42 smithi093 ceph-mon[42070]: pgmap v259: 129 pgs: 1 activating, 1 peering, 127 active+clean; 640 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:43.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:42 smithi160 ceph-mon[43840]: pgmap v259: 129 pgs: 1 activating, 1 peering, 127 active+clean; 640 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:43.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:42 smithi097 ceph-mon[43486]: pgmap v259: 129 pgs: 1 activating, 1 peering, 127 active+clean; 640 KiB data, 78 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s; 15 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:43.349 INFO:tasks.cephadm.ceph_manager.ceph:making progress, resetting timeout 2023-12-14T18:25:43.349 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph pg dump --format=json 2023-12-14T18:25:43.664 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:44.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:43 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2394807993' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-12-14T18:25:44.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:43 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2394807993' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-12-14T18:25:44.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:43 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2394807993' entity='client.admin' cmd=[{"prefix": "status", "format": "json"}]: dispatch 2023-12-14T18:25:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.707+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/913138171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b940ffe00 msgr2=0x7f3b94103dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.707+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/913138171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b940ffe00 0x7f3b94103dd0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f3b800099c0 tx=0x7f3b80009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/913138171 shutdown_connections 2023-12-14T18:25:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/913138171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b94104500 0x7f3b9410c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/913138171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940ff460 0x7f3b940ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/913138171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b940ffe00 0x7f3b94103dd0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/913138171 >> 172.21.15.93:0/913138171 conn(0x7f3b940770a0 msgr2=0x7f3b940774a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:44.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/913138171 shutdown_connections 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/913138171 wait complete. 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 Processor -- start 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 -- start start 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b940ff460 0x7f3b94196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940ffe00 0x7f3b941968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b94104500 0x7f3b9419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9419b340 con 0x7f3b94104500 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9419b4b0 con 0x7f3b940ffe00 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.708+0000 7f3b9b0a0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b9419b670 con 0x7f3b940ff460 2023-12-14T18:25:44.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9963d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b94104500 0x7f3b9419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9963d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b94104500 0x7f3b9419ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35546/0 (socket says 172.21.15.93:35546) 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9963d700 1 -- 172.21.15.93:0/2997517172 learned_addr learned my addr 172.21.15.93:0/2997517172 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b93fff700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940ffe00 0x7f3b941968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b98e3c700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b940ff460 0x7f3b94196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9963d700 1 -- 172.21.15.93:0/2997517172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b940ff460 msgr2=0x7f3b94196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9963d700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b940ff460 0x7f3b94196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9963d700 1 -- 172.21.15.93:0/2997517172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940ffe00 msgr2=0x7f3b941968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9963d700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940ffe00 0x7f3b941968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9963d700 1 -- 172.21.15.93:0/2997517172 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b800095f0 con 0x7f3b94104500 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9963d700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b94104500 0x7f3b9419ad30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c007d70 tx=0x7f3b8c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:44.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b91ffb700 1 -- 172.21.15.93:0/2997517172 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c00ac90 con 0x7f3b94104500 2023-12-14T18:25:44.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b91ffb700 1 -- 172.21.15.93:0/2997517172 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3b8c00adf0 con 0x7f3b94104500 2023-12-14T18:25:44.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b9410f230 con 0x7f3b94104500 2023-12-14T18:25:44.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b91ffb700 1 -- 172.21.15.93:0/2997517172 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b8c0115e0 con 0x7f3b94104500 2023-12-14T18:25:44.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.709+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b9410f0a0 con 0x7f3b94104500 2023-12-14T18:25:44.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.710+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b9410f3c0 con 0x7f3b94104500 2023-12-14T18:25:44.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.711+0000 7f3b91ffb700 1 -- 172.21.15.93:0/2997517172 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3b8c012420 con 0x7f3b94104500 2023-12-14T18:25:44.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.712+0000 7f3b91ffb700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3b840a1260 0x7f3b840a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:44.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.712+0000 7f3b98e3c700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3b840a1260 0x7f3b840a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:44.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.712+0000 7f3b91ffb700 1 -- 172.21.15.93:0/2997517172 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f3b8c0cce40 con 0x7f3b94104500 2023-12-14T18:25:44.717 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.712+0000 7f3b98e3c700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3b840a1260 0x7f3b840a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3b88005950 tx=0x7f3b8800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:44.718 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.716+0000 7f3b91ffb700 1 -- 172.21.15.93:0/2997517172 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3b8c0961d0 con 0x7f3b94104500 2023-12-14T18:25:44.905 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.903+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3b940fcc40 con 0x7f3b840a1260 2023-12-14T18:25:44.917 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.915+0000 7f3b91ffb700 1 -- 172.21.15.93:0/2997517172 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+426834 (secure 0 0 0) 0x7f3b940fcc40 con 0x7f3b840a1260 2023-12-14T18:25:44.923 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:44.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3b840a1260 msgr2=0x7f3b840a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:44.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3b840a1260 0x7f3b840a3710 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f3b88005950 tx=0x7f3b8800b410 comp rx=0 tx=0).stop 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b94104500 msgr2=0x7f3b9419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b94104500 0x7f3b9419ad30 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f3b8c007d70 tx=0x7f3b8c00a340 comp rx=0 tx=0).stop 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 shutdown_connections 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f3b840a1260 0x7f3b840a3710 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3b94104500 0x7f3b9419ad30 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3b940ff460 0x7f3b94196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 --2- 172.21.15.93:0/2997517172 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f3b940ffe00 0x7f3b941968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 >> 172.21.15.93:0/2997517172 conn(0x7f3b940770a0 msgr2=0x7f3b94102fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 shutdown_connections 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:44.918+0000 7f3b9b0a0700 1 -- 172.21.15.93:0/2997517172 wait complete. 2023-12-14T18:25:44.929 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-12-14T18:25:45.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:44 smithi160 ceph-mon[43840]: pgmap v260: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s; 135 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:45.253 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:44 smithi093 ceph-mon[42070]: pgmap v260: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s; 135 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:45.304 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":260,"stamp":"2023-12-14T18:25:43.962464+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655354,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":402,"num_read_kb":323,"num_write":336,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181823,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80412,"kb_used_data":11516,"kb_used_omap":0,"kb_used_meta":68800,"kb_avail":1124861412,"statfs":{"total":1151940427776,"available":1151858085888,"internally_reserved":0,"allocated":11792384,"data_stored":7673555,"data_compressed":317601,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":70451200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":1652,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":51,"num_read_kb":32,"num_write":1,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1087,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.002092"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844182+0000","last_change":"2023-12-14T18:24:36.587133+0000","last_active":"2023-12-14T18:25:37.844182+0000","last_peered":"2023-12-14T18:25:37.844182+0000","last_clean":"2023-12-14T18:25:37.844182+0000","last_became_active":"2023-12-14T18:24:36.586222+0000","last_became_peered":"2023-12-14T18:24:36.586222+0000","last_unstale":"2023-12-14T18:25:37.844182+0000","last_undegraded":"2023-12-14T18:25:37.844182+0000","last_fullsized":"2023-12-14T18:25:37.844182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:35:18.142196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052570+0000","last_change":"2023-12-14T18:24:36.589027+0000","last_active":"2023-12-14T18:25:38.052570+0000","last_peered":"2023-12-14T18:25:38.052570+0000","last_clean":"2023-12-14T18:25:38.052570+0000","last_became_active":"2023-12-14T18:24:36.588785+0000","last_became_peered":"2023-12-14T18:24:36.588785+0000","last_unstale":"2023-12-14T18:25:38.052570+0000","last_undegraded":"2023-12-14T18:25:38.052570+0000","last_fullsized":"2023-12-14T18:25:38.052570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:10:31.188239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052726+0000","last_change":"2023-12-14T18:24:36.584015+0000","last_active":"2023-12-14T18:25:38.052726+0000","last_peered":"2023-12-14T18:25:38.052726+0000","last_clean":"2023-12-14T18:25:38.052726+0000","last_became_active":"2023-12-14T18:24:36.583735+0000","last_became_peered":"2023-12-14T18:24:36.583735+0000","last_unstale":"2023-12-14T18:25:38.052726+0000","last_undegraded":"2023-12-14T18:25:38.052726+0000","last_fullsized":"2023-12-14T18:25:38.052726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:35:38.893225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844897+0000","last_change":"2023-12-14T18:24:36.584553+0000","last_active":"2023-12-14T18:25:37.844897+0000","last_peered":"2023-12-14T18:25:37.844897+0000","last_clean":"2023-12-14T18:25:37.844897+0000","last_became_active":"2023-12-14T18:24:36.583827+0000","last_became_peered":"2023-12-14T18:24:36.583827+0000","last_unstale":"2023-12-14T18:25:37.844897+0000","last_undegraded":"2023-12-14T18:25:37.844897+0000","last_fullsized":"2023-12-14T18:25:37.844897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:25:09.201735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842858+0000","last_change":"2023-12-14T18:24:36.578128+0000","last_active":"2023-12-14T18:25:37.842858+0000","last_peered":"2023-12-14T18:25:37.842858+0000","last_clean":"2023-12-14T18:25:37.842858+0000","last_became_active":"2023-12-14T18:24:36.577784+0000","last_became_peered":"2023-12-14T18:24:36.577784+0000","last_unstale":"2023-12-14T18:25:37.842858+0000","last_undegraded":"2023-12-14T18:25:37.842858+0000","last_fullsized":"2023-12-14T18:25:37.842858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:43:10.548891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053740+0000","last_change":"2023-12-14T18:24:36.588666+0000","last_active":"2023-12-14T18:25:38.053740+0000","last_peered":"2023-12-14T18:25:38.053740+0000","last_clean":"2023-12-14T18:25:38.053740+0000","last_became_active":"2023-12-14T18:24:36.588291+0000","last_became_peered":"2023-12-14T18:24:36.588291+0000","last_unstale":"2023-12-14T18:25:38.053740+0000","last_undegraded":"2023-12-14T18:25:38.053740+0000","last_fullsized":"2023-12-14T18:25:38.053740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:17:14.408821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844209+0000","last_change":"2023-12-14T18:24:36.586637+0000","last_active":"2023-12-14T18:25:37.844209+0000","last_peered":"2023-12-14T18:25:37.844209+0000","last_clean":"2023-12-14T18:25:37.844209+0000","last_became_active":"2023-12-14T18:24:36.583055+0000","last_became_peered":"2023-12-14T18:24:36.583055+0000","last_unstale":"2023-12-14T18:25:37.844209+0000","last_undegraded":"2023-12-14T18:25:37.844209+0000","last_fullsized":"2023-12-14T18:25:37.844209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:18:17.900284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839806+0000","last_change":"2023-12-14T18:24:36.580206+0000","last_active":"2023-12-14T18:25:37.839806+0000","last_peered":"2023-12-14T18:25:37.839806+0000","last_clean":"2023-12-14T18:25:37.839806+0000","last_became_active":"2023-12-14T18:24:36.579904+0000","last_became_peered":"2023-12-14T18:24:36.579904+0000","last_unstale":"2023-12-14T18:25:37.839806+0000","last_undegraded":"2023-12-14T18:25:37.839806+0000","last_fullsized":"2023-12-14T18:25:37.839806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:40:36.607985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841073+0000","last_change":"2023-12-14T18:24:36.589890+0000","last_active":"2023-12-14T18:25:37.841073+0000","last_peered":"2023-12-14T18:25:37.841073+0000","last_clean":"2023-12-14T18:25:37.841073+0000","last_became_active":"2023-12-14T18:24:36.588999+0000","last_became_peered":"2023-12-14T18:24:36.588999+0000","last_unstale":"2023-12-14T18:25:37.841073+0000","last_undegraded":"2023-12-14T18:25:37.841073+0000","last_fullsized":"2023-12-14T18:25:37.841073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:20.169411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.054309+0000","last_change":"2023-12-14T18:24:36.589565+0000","last_active":"2023-12-14T18:25:38.054309+0000","last_peered":"2023-12-14T18:25:38.054309+0000","last_clean":"2023-12-14T18:25:38.054309+0000","last_became_active":"2023-12-14T18:24:36.588822+0000","last_became_peered":"2023-12-14T18:24:36.588822+0000","last_unstale":"2023-12-14T18:25:38.054309+0000","last_undegraded":"2023-12-14T18:25:38.054309+0000","last_fullsized":"2023-12-14T18:25:38.054309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:03:50.712722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844272+0000","last_change":"2023-12-14T18:24:36.593208+0000","last_active":"2023-12-14T18:25:37.844272+0000","last_peered":"2023-12-14T18:25:37.844272+0000","last_clean":"2023-12-14T18:25:37.844272+0000","last_became_active":"2023-12-14T18:24:36.586394+0000","last_became_peered":"2023-12-14T18:24:36.586394+0000","last_unstale":"2023-12-14T18:25:37.844272+0000","last_undegraded":"2023-12-14T18:25:37.844272+0000","last_fullsized":"2023-12-14T18:25:37.844272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:22.984010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.055045+0000","last_change":"2023-12-14T18:25:38.054725+0000","last_active":"2023-12-14T18:25:38.055045+0000","last_peered":"2023-12-14T18:25:38.055045+0000","last_clean":"2023-12-14T18:25:38.055045+0000","last_became_active":"2023-12-14T18:25:38.054505+0000","last_became_peered":"2023-12-14T18:25:38.054505+0000","last_unstale":"2023-12-14T18:25:38.055045+0000","last_undegraded":"2023-12-14T18:25:38.055045+0000","last_fullsized":"2023-12-14T18:25:38.055045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:29:35.537082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843790+0000","last_change":"2023-12-14T18:24:36.584971+0000","last_active":"2023-12-14T18:25:37.843790+0000","last_peered":"2023-12-14T18:25:37.843790+0000","last_clean":"2023-12-14T18:25:37.843790+0000","last_became_active":"2023-12-14T18:24:36.584695+0000","last_became_peered":"2023-12-14T18:24:36.584695+0000","last_unstale":"2023-12-14T18:25:37.843790+0000","last_undegraded":"2023-12-14T18:25:37.843790+0000","last_fullsized":"2023-12-14T18:25:37.843790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:18:22.200476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052278+0000","last_change":"2023-12-14T18:24:36.589692+0000","last_active":"2023-12-14T18:25:38.052278+0000","last_peered":"2023-12-14T18:25:38.052278+0000","last_clean":"2023-12-14T18:25:38.052278+0000","last_became_active":"2023-12-14T18:24:36.589415+0000","last_became_peered":"2023-12-14T18:24:36.589415+0000","last_unstale":"2023-12-14T18:25:38.052278+0000","last_undegraded":"2023-12-14T18:25:38.052278+0000","last_fullsized":"2023-12-14T18:25:38.052278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:34:32.470917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052263+0000","last_change":"2023-12-14T18:24:36.586911+0000","last_active":"2023-12-14T18:25:38.052263+0000","last_peered":"2023-12-14T18:25:38.052263+0000","last_clean":"2023-12-14T18:25:38.052263+0000","last_became_active":"2023-12-14T18:24:36.586600+0000","last_became_peered":"2023-12-14T18:24:36.586600+0000","last_unstale":"2023-12-14T18:25:38.052263+0000","last_undegraded":"2023-12-14T18:25:38.052263+0000","last_fullsized":"2023-12-14T18:25:38.052263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:05:03.670077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840245+0000","last_change":"2023-12-14T18:24:36.584879+0000","last_active":"2023-12-14T18:25:37.840245+0000","last_peered":"2023-12-14T18:25:37.840245+0000","last_clean":"2023-12-14T18:25:37.840245+0000","last_became_active":"2023-12-14T18:24:36.584566+0000","last_became_peered":"2023-12-14T18:24:36.584566+0000","last_unstale":"2023-12-14T18:25:37.840245+0000","last_undegraded":"2023-12-14T18:25:37.840245+0000","last_fullsized":"2023-12-14T18:25:37.840245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:28:45.762443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843862+0000","last_change":"2023-12-14T18:24:36.587420+0000","last_active":"2023-12-14T18:25:37.843862+0000","last_peered":"2023-12-14T18:25:37.843862+0000","last_clean":"2023-12-14T18:25:37.843862+0000","last_became_active":"2023-12-14T18:24:36.586732+0000","last_became_peered":"2023-12-14T18:24:36.586732+0000","last_unstale":"2023-12-14T18:25:37.843862+0000","last_undegraded":"2023-12-14T18:25:37.843862+0000","last_fullsized":"2023-12-14T18:25:37.843862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:05:08.359976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842943+0000","last_change":"2023-12-14T18:24:36.583412+0000","last_active":"2023-12-14T18:25:37.842943+0000","last_peered":"2023-12-14T18:25:37.842943+0000","last_clean":"2023-12-14T18:25:37.842943+0000","last_became_active":"2023-12-14T18:24:36.583022+0000","last_became_peered":"2023-12-14T18:24:36.583022+0000","last_unstale":"2023-12-14T18:25:37.842943+0000","last_undegraded":"2023-12-14T18:25:37.842943+0000","last_fullsized":"2023-12-14T18:25:37.842943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:07:57.550138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841957+0000","last_change":"2023-12-14T18:24:36.575232+0000","last_active":"2023-12-14T18:25:37.841957+0000","last_peered":"2023-12-14T18:25:37.841957+0000","last_clean":"2023-12-14T18:25:37.841957+0000","last_became_active":"2023-12-14T18:24:36.574573+0000","last_became_peered":"2023-12-14T18:24:36.574573+0000","last_unstale":"2023-12-14T18:25:37.841957+0000","last_undegraded":"2023-12-14T18:25:37.841957+0000","last_fullsized":"2023-12-14T18:25:37.841957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:16:28.555112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840943+0000","last_change":"2023-12-14T18:24:40.605077+0000","last_active":"2023-12-14T18:25:37.840943+0000","last_peered":"2023-12-14T18:25:37.840943+0000","last_clean":"2023-12-14T18:25:37.840943+0000","last_became_active":"2023-12-14T18:24:40.604646+0000","last_became_peered":"2023-12-14T18:24:40.604646+0000","last_unstale":"2023-12-14T18:25:37.840943+0000","last_undegraded":"2023-12-14T18:25:37.840943+0000","last_fullsized":"2023-12-14T18:25:37.840943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:59:21.706575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842596+0000","last_change":"2023-12-14T18:24:36.177449+0000","last_active":"2023-12-14T18:25:37.842596+0000","last_peered":"2023-12-14T18:25:37.842596+0000","last_clean":"2023-12-14T18:25:37.842596+0000","last_became_active":"2023-12-14T18:24:36.177114+0000","last_became_peered":"2023-12-14T18:24:36.177114+0000","last_unstale":"2023-12-14T18:25:37.842596+0000","last_undegraded":"2023-12-14T18:25:37.842596+0000","last_fullsized":"2023-12-14T18:25:37.842596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:20.038616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840363+0000","last_change":"2023-12-14T18:24:36.574204+0000","last_active":"2023-12-14T18:25:37.840363+0000","last_peered":"2023-12-14T18:25:37.840363+0000","last_clean":"2023-12-14T18:25:37.840363+0000","last_became_active":"2023-12-14T18:24:36.573923+0000","last_became_peered":"2023-12-14T18:24:36.573923+0000","last_unstale":"2023-12-14T18:25:37.840363+0000","last_undegraded":"2023-12-14T18:25:37.840363+0000","last_fullsized":"2023-12-14T18:25:37.840363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:16:19.375764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843661+0000","last_change":"2023-12-14T18:24:40.600254+0000","last_active":"2023-12-14T18:25:37.843661+0000","last_peered":"2023-12-14T18:25:37.843661+0000","last_clean":"2023-12-14T18:25:37.843661+0000","last_became_active":"2023-12-14T18:24:40.599894+0000","last_became_peered":"2023-12-14T18:24:40.599894+0000","last_unstale":"2023-12-14T18:25:37.843661+0000","last_undegraded":"2023-12-14T18:25:37.843661+0000","last_fullsized":"2023-12-14T18:25:37.843661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:53:10.654777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840496+0000","last_change":"2023-12-14T18:24:36.170109+0000","last_active":"2023-12-14T18:25:37.840496+0000","last_peered":"2023-12-14T18:25:37.840496+0000","last_clean":"2023-12-14T18:25:37.840496+0000","last_became_active":"2023-12-14T18:24:36.169629+0000","last_became_peered":"2023-12-14T18:24:36.169629+0000","last_unstale":"2023-12-14T18:25:37.840496+0000","last_undegraded":"2023-12-14T18:25:37.840496+0000","last_fullsized":"2023-12-14T18:25:37.840496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:15:23.380992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052504+0000","last_change":"2023-12-14T18:24:36.582757+0000","last_active":"2023-12-14T18:25:38.052504+0000","last_peered":"2023-12-14T18:25:38.052504+0000","last_clean":"2023-12-14T18:25:38.052504+0000","last_became_active":"2023-12-14T18:24:36.582457+0000","last_became_peered":"2023-12-14T18:24:36.582457+0000","last_unstale":"2023-12-14T18:25:38.052504+0000","last_undegraded":"2023-12-14T18:25:38.052504+0000","last_fullsized":"2023-12-14T18:25:38.052504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:35:13.993694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845045+0000","last_change":"2023-12-14T18:24:40.599695+0000","last_active":"2023-12-14T18:25:37.845045+0000","last_peered":"2023-12-14T18:25:37.845045+0000","last_clean":"2023-12-14T18:25:37.845045+0000","last_became_active":"2023-12-14T18:24:40.599239+0000","last_became_peered":"2023-12-14T18:24:40.599239+0000","last_unstale":"2023-12-14T18:25:37.845045+0000","last_undegraded":"2023-12-14T18:25:37.845045+0000","last_fullsized":"2023-12-14T18:25:37.845045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:54:19.445064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846330+0000","last_change":"2023-12-14T18:24:36.177946+0000","last_active":"2023-12-14T18:25:37.846330+0000","last_peered":"2023-12-14T18:25:37.846330+0000","last_clean":"2023-12-14T18:25:37.846330+0000","last_became_active":"2023-12-14T18:24:36.177480+0000","last_became_peered":"2023-12-14T18:24:36.177480+0000","last_unstale":"2023-12-14T18:25:37.846330+0000","last_undegraded":"2023-12-14T18:25:37.846330+0000","last_fullsized":"2023-12-14T18:25:37.846330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:58:29.278977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840508+0000","last_change":"2023-12-14T18:24:36.587951+0000","last_active":"2023-12-14T18:25:37.840508+0000","last_peered":"2023-12-14T18:25:37.840508+0000","last_clean":"2023-12-14T18:25:37.840508+0000","last_became_active":"2023-12-14T18:24:36.587539+0000","last_became_peered":"2023-12-14T18:24:36.587539+0000","last_unstale":"2023-12-14T18:25:37.840508+0000","last_undegraded":"2023-12-14T18:25:37.840508+0000","last_fullsized":"2023-12-14T18:25:37.840508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:18:40.567638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843868+0000","last_change":"2023-12-14T18:24:40.600008+0000","last_active":"2023-12-14T18:25:37.843868+0000","last_peered":"2023-12-14T18:25:37.843868+0000","last_clean":"2023-12-14T18:25:37.843868+0000","last_became_active":"2023-12-14T18:24:40.599596+0000","last_became_peered":"2023-12-14T18:24:40.599596+0000","last_unstale":"2023-12-14T18:25:37.843868+0000","last_undegraded":"2023-12-14T18:25:37.843868+0000","last_fullsized":"2023-12-14T18:25:37.843868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:06:21.571807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845392+0000","last_change":"2023-12-14T18:24:36.176569+0000","last_active":"2023-12-14T18:25:37.845392+0000","last_peered":"2023-12-14T18:25:37.845392+0000","last_clean":"2023-12-14T18:25:37.845392+0000","last_became_active":"2023-12-14T18:24:36.176043+0000","last_became_peered":"2023-12-14T18:24:36.176043+0000","last_unstale":"2023-12-14T18:25:37.845392+0000","last_undegraded":"2023-12-14T18:25:37.845392+0000","last_fullsized":"2023-12-14T18:25:37.845392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:37:45.325365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.847594+0000","last_change":"2023-12-14T18:24:36.593761+0000","last_active":"2023-12-14T18:25:37.847594+0000","last_peered":"2023-12-14T18:25:37.847594+0000","last_clean":"2023-12-14T18:25:37.847594+0000","last_became_active":"2023-12-14T18:24:36.593452+0000","last_became_peered":"2023-12-14T18:24:36.593452+0000","last_unstale":"2023-12-14T18:25:37.847594+0000","last_undegraded":"2023-12-14T18:25:37.847594+0000","last_fullsized":"2023-12-14T18:25:37.847594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:05:51.179026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841661+0000","last_change":"2023-12-14T18:24:40.598064+0000","last_active":"2023-12-14T18:25:37.841661+0000","last_peered":"2023-12-14T18:25:37.841661+0000","last_clean":"2023-12-14T18:25:37.841661+0000","last_became_active":"2023-12-14T18:24:40.597672+0000","last_became_peered":"2023-12-14T18:24:40.597672+0000","last_unstale":"2023-12-14T18:25:37.841661+0000","last_undegraded":"2023-12-14T18:25:37.841661+0000","last_fullsized":"2023-12-14T18:25:37.841661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:19:47.256702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":89,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:40.856321+0000","last_change":"2023-12-14T18:25:37.854647+0000","last_active":"2023-12-14T18:25:40.856321+0000","last_peered":"2023-12-14T18:25:40.856321+0000","last_clean":"2023-12-14T18:25:40.856321+0000","last_became_active":"2023-12-14T18:25:37.848559+0000","last_became_peered":"2023-12-14T18:25:37.848559+0000","last_unstale":"2023-12-14T18:25:40.856321+0000","last_undegraded":"2023-12-14T18:25:40.856321+0000","last_fullsized":"2023-12-14T18:25:40.856321+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:03.962836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843816+0000","last_change":"2023-12-14T18:24:36.583687+0000","last_active":"2023-12-14T18:25:37.843816+0000","last_peered":"2023-12-14T18:25:37.843816+0000","last_clean":"2023-12-14T18:25:37.843816+0000","last_became_active":"2023-12-14T18:24:36.583047+0000","last_became_peered":"2023-12-14T18:24:36.583047+0000","last_unstale":"2023-12-14T18:25:37.843816+0000","last_undegraded":"2023-12-14T18:25:37.843816+0000","last_fullsized":"2023-12-14T18:25:37.843816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:28.910079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841353+0000","last_change":"2023-12-14T18:24:40.605217+0000","last_active":"2023-12-14T18:25:37.841353+0000","last_peered":"2023-12-14T18:25:37.841353+0000","last_clean":"2023-12-14T18:25:37.841353+0000","last_became_active":"2023-12-14T18:24:40.604807+0000","last_became_peered":"2023-12-14T18:24:40.604807+0000","last_unstale":"2023-12-14T18:25:37.841353+0000","last_undegraded":"2023-12-14T18:25:37.841353+0000","last_fullsized":"2023-12-14T18:25:37.841353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:00:15.863206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053667+0000","last_change":"2023-12-14T18:24:36.174875+0000","last_active":"2023-12-14T18:25:38.053667+0000","last_peered":"2023-12-14T18:25:38.053667+0000","last_clean":"2023-12-14T18:25:38.053667+0000","last_became_active":"2023-12-14T18:24:36.173984+0000","last_became_peered":"2023-12-14T18:24:36.173984+0000","last_unstale":"2023-12-14T18:25:38.053667+0000","last_undegraded":"2023-12-14T18:25:38.053667+0000","last_fullsized":"2023-12-14T18:25:38.053667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:09:45.465327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846977+0000","last_change":"2023-12-14T18:24:36.580563+0000","last_active":"2023-12-14T18:25:37.846977+0000","last_peered":"2023-12-14T18:25:37.846977+0000","last_clean":"2023-12-14T18:25:37.846977+0000","last_became_active":"2023-12-14T18:24:36.580196+0000","last_became_peered":"2023-12-14T18:24:36.580196+0000","last_unstale":"2023-12-14T18:25:37.846977+0000","last_undegraded":"2023-12-14T18:25:37.846977+0000","last_fullsized":"2023-12-14T18:25:37.846977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:32:02.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843700+0000","last_change":"2023-12-14T18:24:40.601223+0000","last_active":"2023-12-14T18:25:37.843700+0000","last_peered":"2023-12-14T18:25:37.843700+0000","last_clean":"2023-12-14T18:25:37.843700+0000","last_became_active":"2023-12-14T18:24:40.600795+0000","last_became_peered":"2023-12-14T18:24:40.600795+0000","last_unstale":"2023-12-14T18:25:37.843700+0000","last_undegraded":"2023-12-14T18:25:37.843700+0000","last_fullsized":"2023-12-14T18:25:37.843700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:31.442312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843493+0000","last_change":"2023-12-14T18:24:36.173600+0000","last_active":"2023-12-14T18:25:37.843493+0000","last_peered":"2023-12-14T18:25:37.843493+0000","last_clean":"2023-12-14T18:25:37.843493+0000","last_became_active":"2023-12-14T18:24:36.173272+0000","last_became_peered":"2023-12-14T18:24:36.173272+0000","last_unstale":"2023-12-14T18:25:37.843493+0000","last_undegraded":"2023-12-14T18:25:37.843493+0000","last_fullsized":"2023-12-14T18:25:37.843493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:36:35.523553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839726+0000","last_change":"2023-12-14T18:24:36.575060+0000","last_active":"2023-12-14T18:25:37.839726+0000","last_peered":"2023-12-14T18:25:37.839726+0000","last_clean":"2023-12-14T18:25:37.839726+0000","last_became_active":"2023-12-14T18:24:36.574749+0000","last_became_peered":"2023-12-14T18:24:36.574749+0000","last_unstale":"2023-12-14T18:25:37.839726+0000","last_undegraded":"2023-12-14T18:25:37.839726+0000","last_fullsized":"2023-12-14T18:25:37.839726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:23:49.611073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052543+0000","last_change":"2023-12-14T18:24:40.605431+0000","last_active":"2023-12-14T18:25:38.052543+0000","last_peered":"2023-12-14T18:25:38.052543+0000","last_clean":"2023-12-14T18:25:38.052543+0000","last_became_active":"2023-12-14T18:24:40.604951+0000","last_became_peered":"2023-12-14T18:24:40.604951+0000","last_unstale":"2023-12-14T18:25:38.052543+0000","last_undegraded":"2023-12-14T18:25:38.052543+0000","last_fullsized":"2023-12-14T18:25:38.052543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:52:55.859566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845146+0000","last_change":"2023-12-14T18:24:36.175493+0000","last_active":"2023-12-14T18:25:37.845146+0000","last_peered":"2023-12-14T18:25:37.845146+0000","last_clean":"2023-12-14T18:25:37.845146+0000","last_became_active":"2023-12-14T18:24:36.175205+0000","last_became_peered":"2023-12-14T18:24:36.175205+0000","last_unstale":"2023-12-14T18:25:37.845146+0000","last_undegraded":"2023-12-14T18:25:37.845146+0000","last_fullsized":"2023-12-14T18:25:37.845146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:15:17.424259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842088+0000","last_change":"2023-12-14T18:24:36.587218+0000","last_active":"2023-12-14T18:25:37.842088+0000","last_peered":"2023-12-14T18:25:37.842088+0000","last_clean":"2023-12-14T18:25:37.842088+0000","last_became_active":"2023-12-14T18:24:36.586910+0000","last_became_peered":"2023-12-14T18:24:36.586910+0000","last_unstale":"2023-12-14T18:25:37.842088+0000","last_undegraded":"2023-12-14T18:25:37.842088+0000","last_fullsized":"2023-12-14T18:25:37.842088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:27.244227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839960+0000","last_change":"2023-12-14T18:24:40.598409+0000","last_active":"2023-12-14T18:25:37.839960+0000","last_peered":"2023-12-14T18:25:37.839960+0000","last_clean":"2023-12-14T18:25:37.839960+0000","last_became_active":"2023-12-14T18:24:40.598011+0000","last_became_peered":"2023-12-14T18:24:40.598011+0000","last_unstale":"2023-12-14T18:25:37.839960+0000","last_undegraded":"2023-12-14T18:25:37.839960+0000","last_fullsized":"2023-12-14T18:25:37.839960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:02:12.191033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":70,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:39.854530+0000","last_change":"2023-12-14T18:24:36.172277+0000","last_active":"2023-12-14T18:25:39.854530+0000","last_peered":"2023-12-14T18:25:39.854530+0000","last_clean":"2023-12-14T18:25:39.854530+0000","last_became_active":"2023-12-14T18:24:36.171907+0000","last_became_peered":"2023-12-14T18:24:36.171907+0000","last_unstale":"2023-12-14T18:25:39.854530+0000","last_undegraded":"2023-12-14T18:25:39.854530+0000","last_fullsized":"2023-12-14T18:25:39.854530+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:51:05.472295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":41,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840975+0000","last_change":"2023-12-14T18:24:36.584783+0000","last_active":"2023-12-14T18:25:37.840975+0000","last_peered":"2023-12-14T18:25:37.840975+0000","last_clean":"2023-12-14T18:25:37.840975+0000","last_became_active":"2023-12-14T18:24:36.584424+0000","last_became_peered":"2023-12-14T18:24:36.584424+0000","last_unstale":"2023-12-14T18:25:37.840975+0000","last_undegraded":"2023-12-14T18:25:37.840975+0000","last_fullsized":"2023-12-14T18:25:37.840975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:21.204303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841959+0000","last_change":"2023-12-14T18:24:40.603176+0000","last_active":"2023-12-14T18:25:37.841959+0000","last_peered":"2023-12-14T18:25:37.841959+0000","last_clean":"2023-12-14T18:25:37.841959+0000","last_became_active":"2023-12-14T18:24:40.602868+0000","last_became_peered":"2023-12-14T18:24:40.602868+0000","last_unstale":"2023-12-14T18:25:37.841959+0000","last_undegraded":"2023-12-14T18:25:37.841959+0000","last_fullsized":"2023-12-14T18:25:37.841959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:15:57.475854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844517+0000","last_change":"2023-12-14T18:24:36.177638+0000","last_active":"2023-12-14T18:25:37.844517+0000","last_peered":"2023-12-14T18:25:37.844517+0000","last_clean":"2023-12-14T18:25:37.844517+0000","last_became_active":"2023-12-14T18:24:36.177335+0000","last_became_peered":"2023-12-14T18:24:36.177335+0000","last_unstale":"2023-12-14T18:25:37.844517+0000","last_undegraded":"2023-12-14T18:25:37.844517+0000","last_fullsized":"2023-12-14T18:25:37.844517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:29:24.204595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052948+0000","last_change":"2023-12-14T18:24:36.577508+0000","last_active":"2023-12-14T18:25:38.052948+0000","last_peered":"2023-12-14T18:25:38.052948+0000","last_clean":"2023-12-14T18:25:38.052948+0000","last_became_active":"2023-12-14T18:24:36.577196+0000","last_became_peered":"2023-12-14T18:24:36.577196+0000","last_unstale":"2023-12-14T18:25:38.052948+0000","last_undegraded":"2023-12-14T18:25:38.052948+0000","last_fullsized":"2023-12-14T18:25:38.052948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:55:24.227563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845217+0000","last_change":"2023-12-14T18:24:40.594816+0000","last_active":"2023-12-14T18:25:37.845217+0000","last_peered":"2023-12-14T18:25:37.845217+0000","last_clean":"2023-12-14T18:25:37.845217+0000","last_became_active":"2023-12-14T18:24:40.594371+0000","last_became_peered":"2023-12-14T18:24:40.594371+0000","last_unstale":"2023-12-14T18:25:37.845217+0000","last_undegraded":"2023-12-14T18:25:37.845217+0000","last_fullsized":"2023-12-14T18:25:37.845217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:02:50.352263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.847468+0000","last_change":"2023-12-14T18:24:36.174238+0000","last_active":"2023-12-14T18:25:37.847468+0000","last_peered":"2023-12-14T18:25:37.847468+0000","last_clean":"2023-12-14T18:25:37.847468+0000","last_became_active":"2023-12-14T18:24:36.173373+0000","last_became_peered":"2023-12-14T18:24:36.173373+0000","last_unstale":"2023-12-14T18:25:37.847468+0000","last_undegraded":"2023-12-14T18:25:37.847468+0000","last_fullsized":"2023-12-14T18:25:37.847468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:48:08.262361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840699+0000","last_change":"2023-12-14T18:24:36.588589+0000","last_active":"2023-12-14T18:25:37.840699+0000","last_peered":"2023-12-14T18:25:37.840699+0000","last_clean":"2023-12-14T18:25:37.840699+0000","last_became_active":"2023-12-14T18:24:36.588269+0000","last_became_peered":"2023-12-14T18:24:36.588269+0000","last_unstale":"2023-12-14T18:25:37.840699+0000","last_undegraded":"2023-12-14T18:25:37.840699+0000","last_fullsized":"2023-12-14T18:25:37.840699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:55:55.150514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842244+0000","last_change":"2023-12-14T18:24:40.601333+0000","last_active":"2023-12-14T18:25:37.842244+0000","last_peered":"2023-12-14T18:25:37.842244+0000","last_clean":"2023-12-14T18:25:37.842244+0000","last_became_active":"2023-12-14T18:24:40.600966+0000","last_became_peered":"2023-12-14T18:24:40.600966+0000","last_unstale":"2023-12-14T18:25:37.842244+0000","last_undegraded":"2023-12-14T18:25:37.842244+0000","last_fullsized":"2023-12-14T18:25:37.842244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:30:57.057783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":147,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:40.852647+0000","last_change":"2023-12-14T18:24:36.177299+0000","last_active":"2023-12-14T18:25:40.852647+0000","last_peered":"2023-12-14T18:25:40.852647+0000","last_clean":"2023-12-14T18:25:40.852647+0000","last_became_active":"2023-12-14T18:24:36.176971+0000","last_became_peered":"2023-12-14T18:24:36.176971+0000","last_unstale":"2023-12-14T18:25:40.852647+0000","last_undegraded":"2023-12-14T18:25:40.852647+0000","last_fullsized":"2023-12-14T18:25:40.852647+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:09:55.682617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60836,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":90,"num_read_kb":104,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846054+0000","last_change":"2023-12-14T18:24:36.586315+0000","last_active":"2023-12-14T18:25:37.846054+0000","last_peered":"2023-12-14T18:25:37.846054+0000","last_clean":"2023-12-14T18:25:37.846054+0000","last_became_active":"2023-12-14T18:24:36.586000+0000","last_became_peered":"2023-12-14T18:24:36.586000+0000","last_unstale":"2023-12-14T18:25:37.846054+0000","last_undegraded":"2023-12-14T18:25:37.846054+0000","last_fullsized":"2023-12-14T18:25:37.846054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:55:32.691488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.847864+0000","last_change":"2023-12-14T18:24:40.596973+0000","last_active":"2023-12-14T18:25:37.847864+0000","last_peered":"2023-12-14T18:25:37.847864+0000","last_clean":"2023-12-14T18:25:37.847864+0000","last_became_active":"2023-12-14T18:24:40.596355+0000","last_became_peered":"2023-12-14T18:24:40.596355+0000","last_unstale":"2023-12-14T18:25:37.847864+0000","last_undegraded":"2023-12-14T18:25:37.847864+0000","last_fullsized":"2023-12-14T18:25:37.847864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:05:47.138776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840994+0000","last_change":"2023-12-14T18:24:36.175287+0000","last_active":"2023-12-14T18:25:37.840994+0000","last_peered":"2023-12-14T18:25:37.840994+0000","last_clean":"2023-12-14T18:25:37.840994+0000","last_became_active":"2023-12-14T18:24:36.174952+0000","last_became_peered":"2023-12-14T18:24:36.174952+0000","last_unstale":"2023-12-14T18:25:37.840994+0000","last_undegraded":"2023-12-14T18:25:37.840994+0000","last_fullsized":"2023-12-14T18:25:37.840994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:56:55.084380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843323+0000","last_change":"2023-12-14T18:24:36.582314+0000","last_active":"2023-12-14T18:25:37.843323+0000","last_peered":"2023-12-14T18:25:37.843323+0000","last_clean":"2023-12-14T18:25:37.843323+0000","last_became_active":"2023-12-14T18:24:36.581963+0000","last_became_peered":"2023-12-14T18:24:36.581963+0000","last_unstale":"2023-12-14T18:25:37.843323+0000","last_undegraded":"2023-12-14T18:25:37.843323+0000","last_fullsized":"2023-12-14T18:25:37.843323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:01:58.910449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843110+0000","last_change":"2023-12-14T18:24:40.599979+0000","last_active":"2023-12-14T18:25:37.843110+0000","last_peered":"2023-12-14T18:25:37.843110+0000","last_clean":"2023-12-14T18:25:37.843110+0000","last_became_active":"2023-12-14T18:24:40.599258+0000","last_became_peered":"2023-12-14T18:24:40.599258+0000","last_unstale":"2023-12-14T18:25:37.843110+0000","last_undegraded":"2023-12-14T18:25:37.843110+0000","last_fullsized":"2023-12-14T18:25:37.843110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:16:48.809775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846703+0000","last_change":"2023-12-14T18:24:36.175768+0000","last_active":"2023-12-14T18:25:37.846703+0000","last_peered":"2023-12-14T18:25:37.846703+0000","last_clean":"2023-12-14T18:25:37.846703+0000","last_became_active":"2023-12-14T18:24:36.174973+0000","last_became_peered":"2023-12-14T18:24:36.174973+0000","last_unstale":"2023-12-14T18:25:37.846703+0000","last_undegraded":"2023-12-14T18:25:37.846703+0000","last_fullsized":"2023-12-14T18:25:37.846703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:41:10.229183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844716+0000","last_change":"2023-12-14T18:24:36.591856+0000","last_active":"2023-12-14T18:25:37.844716+0000","last_peered":"2023-12-14T18:25:37.844716+0000","last_clean":"2023-12-14T18:25:37.844716+0000","last_became_active":"2023-12-14T18:24:36.591554+0000","last_became_peered":"2023-12-14T18:24:36.591554+0000","last_unstale":"2023-12-14T18:25:37.844716+0000","last_undegraded":"2023-12-14T18:25:37.844716+0000","last_fullsized":"2023-12-14T18:25:37.844716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:07:38.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842256+0000","last_change":"2023-12-14T18:24:40.603794+0000","last_active":"2023-12-14T18:25:37.842256+0000","last_peered":"2023-12-14T18:25:37.842256+0000","last_clean":"2023-12-14T18:25:37.842256+0000","last_became_active":"2023-12-14T18:24:40.603311+0000","last_became_peered":"2023-12-14T18:24:40.603311+0000","last_unstale":"2023-12-14T18:25:37.842256+0000","last_undegraded":"2023-12-14T18:25:37.842256+0000","last_fullsized":"2023-12-14T18:25:37.842256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:27:38.742031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":70,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.839546+0000","last_change":"2023-12-14T18:24:36.177411+0000","last_active":"2023-12-14T18:25:38.839546+0000","last_peered":"2023-12-14T18:25:38.839546+0000","last_clean":"2023-12-14T18:25:38.839546+0000","last_became_active":"2023-12-14T18:24:36.177047+0000","last_became_peered":"2023-12-14T18:24:36.177047+0000","last_unstale":"2023-12-14T18:25:38.839546+0000","last_undegraded":"2023-12-14T18:25:38.839546+0000","last_fullsized":"2023-12-14T18:25:38.839546+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:25:34.315245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":42,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.054628+0000","last_change":"2023-12-14T18:25:38.054628+0000","last_active":"2023-12-14T18:25:38.054628+0000","last_peered":"2023-12-14T18:25:38.054628+0000","last_clean":"2023-12-14T18:25:38.054628+0000","last_became_active":"2023-12-14T18:25:38.053719+0000","last_became_peered":"2023-12-14T18:25:38.053719+0000","last_unstale":"2023-12-14T18:25:38.054628+0000","last_undegraded":"2023-12-14T18:25:38.054628+0000","last_fullsized":"2023-12-14T18:25:38.054628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:19:00.666432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844240+0000","last_change":"2023-12-14T18:24:40.607042+0000","last_active":"2023-12-14T18:25:37.844240+0000","last_peered":"2023-12-14T18:25:37.844240+0000","last_clean":"2023-12-14T18:25:37.844240+0000","last_became_active":"2023-12-14T18:24:40.606563+0000","last_became_peered":"2023-12-14T18:24:40.606563+0000","last_unstale":"2023-12-14T18:25:37.844240+0000","last_undegraded":"2023-12-14T18:25:37.844240+0000","last_fullsized":"2023-12-14T18:25:37.844240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:17:48.751456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842804+0000","last_change":"2023-12-14T18:24:36.171994+0000","last_active":"2023-12-14T18:25:37.842804+0000","last_peered":"2023-12-14T18:25:37.842804+0000","last_clean":"2023-12-14T18:25:37.842804+0000","last_became_active":"2023-12-14T18:24:36.171661+0000","last_became_peered":"2023-12-14T18:24:36.171661+0000","last_unstale":"2023-12-14T18:25:37.842804+0000","last_undegraded":"2023-12-14T18:25:37.842804+0000","last_fullsized":"2023-12-14T18:25:37.842804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:00:31.916889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845012+0000","last_change":"2023-12-14T18:24:36.577818+0000","last_active":"2023-12-14T18:25:37.845012+0000","last_peered":"2023-12-14T18:25:37.845012+0000","last_clean":"2023-12-14T18:25:37.845012+0000","last_became_active":"2023-12-14T18:24:36.577356+0000","last_became_peered":"2023-12-14T18:24:36.577356+0000","last_unstale":"2023-12-14T18:25:37.845012+0000","last_undegraded":"2023-12-14T18:25:37.845012+0000","last_fullsized":"2023-12-14T18:25:37.845012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:27:12.642108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844848+0000","last_change":"2023-12-14T18:24:40.601995+0000","last_active":"2023-12-14T18:25:37.844848+0000","last_peered":"2023-12-14T18:25:37.844848+0000","last_clean":"2023-12-14T18:25:37.844848+0000","last_became_active":"2023-12-14T18:24:40.601571+0000","last_became_peered":"2023-12-14T18:24:40.601571+0000","last_unstale":"2023-12-14T18:25:37.844848+0000","last_undegraded":"2023-12-14T18:25:37.844848+0000","last_fullsized":"2023-12-14T18:25:37.844848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:37:19.308982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846519+0000","last_change":"2023-12-14T18:24:36.168661+0000","last_active":"2023-12-14T18:25:37.846519+0000","last_peered":"2023-12-14T18:25:37.846519+0000","last_clean":"2023-12-14T18:25:37.846519+0000","last_became_active":"2023-12-14T18:24:36.168299+0000","last_became_peered":"2023-12-14T18:24:36.168299+0000","last_unstale":"2023-12-14T18:25:37.846519+0000","last_undegraded":"2023-12-14T18:25:37.846519+0000","last_fullsized":"2023-12-14T18:25:37.846519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:54.195843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843080+0000","last_change":"2023-12-14T18:24:36.578417+0000","last_active":"2023-12-14T18:25:37.843080+0000","last_peered":"2023-12-14T18:25:37.843080+0000","last_clean":"2023-12-14T18:25:37.843080+0000","last_became_active":"2023-12-14T18:24:36.578080+0000","last_became_peered":"2023-12-14T18:24:36.578080+0000","last_unstale":"2023-12-14T18:25:37.843080+0000","last_undegraded":"2023-12-14T18:25:37.843080+0000","last_fullsized":"2023-12-14T18:25:37.843080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:55:52.265641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.055456+0000","last_change":"2023-12-14T18:25:38.055456+0000","last_active":"2023-12-14T18:25:38.055456+0000","last_peered":"2023-12-14T18:25:38.055456+0000","last_clean":"2023-12-14T18:25:38.055456+0000","last_became_active":"2023-12-14T18:25:38.055025+0000","last_became_peered":"2023-12-14T18:25:38.055025+0000","last_unstale":"2023-12-14T18:25:38.055456+0000","last_undegraded":"2023-12-14T18:25:38.055456+0000","last_fullsized":"2023-12-14T18:25:38.055456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:18:47.589447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842458+0000","last_change":"2023-12-14T18:24:36.170489+0000","last_active":"2023-12-14T18:25:37.842458+0000","last_peered":"2023-12-14T18:25:37.842458+0000","last_clean":"2023-12-14T18:25:37.842458+0000","last_became_active":"2023-12-14T18:24:36.170161+0000","last_became_peered":"2023-12-14T18:24:36.170161+0000","last_unstale":"2023-12-14T18:25:37.842458+0000","last_undegraded":"2023-12-14T18:25:37.842458+0000","last_fullsized":"2023-12-14T18:25:37.842458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:49:25.035687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053967+0000","last_change":"2023-12-14T18:24:36.591094+0000","last_active":"2023-12-14T18:25:38.053967+0000","last_peered":"2023-12-14T18:25:38.053967+0000","last_clean":"2023-12-14T18:25:38.053967+0000","last_became_active":"2023-12-14T18:24:36.590779+0000","last_became_peered":"2023-12-14T18:24:36.590779+0000","last_unstale":"2023-12-14T18:25:38.053967+0000","last_undegraded":"2023-12-14T18:25:38.053967+0000","last_fullsized":"2023-12-14T18:25:38.053967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:58:59.945071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843917+0000","last_change":"2023-12-14T18:24:40.604125+0000","last_active":"2023-12-14T18:25:37.843917+0000","last_peered":"2023-12-14T18:25:37.843917+0000","last_clean":"2023-12-14T18:25:37.843917+0000","last_became_active":"2023-12-14T18:24:40.603242+0000","last_became_peered":"2023-12-14T18:24:40.603242+0000","last_unstale":"2023-12-14T18:25:37.843917+0000","last_undegraded":"2023-12-14T18:25:37.843917+0000","last_fullsized":"2023-12-14T18:25:37.843917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:48:08.760578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845182+0000","last_change":"2023-12-14T18:24:36.177567+0000","last_active":"2023-12-14T18:25:37.845182+0000","last_peered":"2023-12-14T18:25:37.845182+0000","last_clean":"2023-12-14T18:25:37.845182+0000","last_became_active":"2023-12-14T18:24:36.176949+0000","last_became_peered":"2023-12-14T18:24:36.176949+0000","last_unstale":"2023-12-14T18:25:37.845182+0000","last_undegraded":"2023-12-14T18:25:37.845182+0000","last_fullsized":"2023-12-14T18:25:37.845182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:41:12.145991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053915+0000","last_change":"2023-12-14T18:24:36.594224+0000","last_active":"2023-12-14T18:25:38.053915+0000","last_peered":"2023-12-14T18:25:38.053915+0000","last_clean":"2023-12-14T18:25:38.053915+0000","last_became_active":"2023-12-14T18:24:36.593859+0000","last_became_peered":"2023-12-14T18:24:36.593859+0000","last_unstale":"2023-12-14T18:25:38.053915+0000","last_undegraded":"2023-12-14T18:25:38.053915+0000","last_fullsized":"2023-12-14T18:25:38.053915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:12:34.937171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844213+0000","last_change":"2023-12-14T18:24:40.771814+0000","last_active":"2023-12-14T18:25:37.844213+0000","last_peered":"2023-12-14T18:25:37.844213+0000","last_clean":"2023-12-14T18:25:37.844213+0000","last_became_active":"2023-12-14T18:24:40.771387+0000","last_became_peered":"2023-12-14T18:24:40.771387+0000","last_unstale":"2023-12-14T18:25:37.844213+0000","last_undegraded":"2023-12-14T18:25:37.844213+0000","last_fullsized":"2023-12-14T18:25:37.844213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:50:15.946615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845399+0000","last_change":"2023-12-14T18:24:36.588826+0000","last_active":"2023-12-14T18:25:37.845399+0000","last_peered":"2023-12-14T18:25:37.845399+0000","last_clean":"2023-12-14T18:25:37.845399+0000","last_became_active":"2023-12-14T18:24:36.588320+0000","last_became_peered":"2023-12-14T18:24:36.588320+0000","last_unstale":"2023-12-14T18:25:37.845399+0000","last_undegraded":"2023-12-14T18:25:37.845399+0000","last_fullsized":"2023-12-14T18:25:37.845399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:41:32.298321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":70,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:39.840517+0000","last_change":"2023-12-14T18:24:36.173995+0000","last_active":"2023-12-14T18:25:39.840517+0000","last_peered":"2023-12-14T18:25:39.840517+0000","last_clean":"2023-12-14T18:25:39.840517+0000","last_became_active":"2023-12-14T18:24:36.173381+0000","last_became_peered":"2023-12-14T18:24:36.173381+0000","last_unstale":"2023-12-14T18:25:39.840517+0000","last_undegraded":"2023-12-14T18:25:39.840517+0000","last_fullsized":"2023-12-14T18:25:39.840517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:05:11.493177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":40,"num_read_kb":40,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844039+0000","last_change":"2023-12-14T18:24:40.605099+0000","last_active":"2023-12-14T18:25:37.844039+0000","last_peered":"2023-12-14T18:25:37.844039+0000","last_clean":"2023-12-14T18:25:37.844039+0000","last_became_active":"2023-12-14T18:24:40.604773+0000","last_became_peered":"2023-12-14T18:24:40.604773+0000","last_unstale":"2023-12-14T18:25:37.844039+0000","last_undegraded":"2023-12-14T18:25:37.844039+0000","last_fullsized":"2023-12-14T18:25:37.844039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:03.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839914+0000","last_change":"2023-12-14T18:24:36.574993+0000","last_active":"2023-12-14T18:25:37.839914+0000","last_peered":"2023-12-14T18:25:37.839914+0000","last_clean":"2023-12-14T18:25:37.839914+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:25:37.839914+0000","last_undegraded":"2023-12-14T18:25:37.839914+0000","last_fullsized":"2023-12-14T18:25:37.839914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:32:30.973673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845214+0000","last_change":"2023-12-14T18:24:36.170061+0000","last_active":"2023-12-14T18:25:37.845214+0000","last_peered":"2023-12-14T18:25:37.845214+0000","last_clean":"2023-12-14T18:25:37.845214+0000","last_became_active":"2023-12-14T18:24:36.169652+0000","last_became_peered":"2023-12-14T18:24:36.169652+0000","last_unstale":"2023-12-14T18:25:37.845214+0000","last_undegraded":"2023-12-14T18:25:37.845214+0000","last_fullsized":"2023-12-14T18:25:37.845214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:46:50.366062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843576+0000","last_change":"2023-12-14T18:23:49.076644+0000","last_active":"2023-12-14T18:25:37.843576+0000","last_peered":"2023-12-14T18:25:37.843576+0000","last_clean":"2023-12-14T18:25:37.843576+0000","last_became_active":"2023-12-14T18:23:49.076263+0000","last_became_peered":"2023-12-14T18:23:49.076263+0000","last_unstale":"2023-12-14T18:25:37.843576+0000","last_undegraded":"2023-12-14T18:25:37.843576+0000","last_fullsized":"2023-12-14T18:25:37.843576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_clean_scrub_stamp":"2023-12-14T18:21:24.715131+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:26:53.561034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.848093+0000","last_change":"2023-12-14T18:24:40.772015+0000","last_active":"2023-12-14T18:25:37.848093+0000","last_peered":"2023-12-14T18:25:37.848093+0000","last_clean":"2023-12-14T18:25:37.848093+0000","last_became_active":"2023-12-14T18:24:40.771497+0000","last_became_peered":"2023-12-14T18:24:40.771497+0000","last_unstale":"2023-12-14T18:25:37.848093+0000","last_undegraded":"2023-12-14T18:25:37.848093+0000","last_fullsized":"2023-12-14T18:25:37.848093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:50:50.964766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842573+0000","last_change":"2023-12-14T18:24:36.584402+0000","last_active":"2023-12-14T18:25:37.842573+0000","last_peered":"2023-12-14T18:25:37.842573+0000","last_clean":"2023-12-14T18:25:37.842573+0000","last_became_active":"2023-12-14T18:24:36.584177+0000","last_became_peered":"2023-12-14T18:24:36.584177+0000","last_unstale":"2023-12-14T18:25:37.842573+0000","last_undegraded":"2023-12-14T18:25:37.842573+0000","last_fullsized":"2023-12-14T18:25:37.842573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:44:22.049474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843698+0000","last_change":"2023-12-14T18:24:36.175017+0000","last_active":"2023-12-14T18:25:37.843698+0000","last_peered":"2023-12-14T18:25:37.843698+0000","last_clean":"2023-12-14T18:25:37.843698+0000","last_became_active":"2023-12-14T18:24:36.174639+0000","last_became_peered":"2023-12-14T18:24:36.174639+0000","last_unstale":"2023-12-14T18:25:37.843698+0000","last_undegraded":"2023-12-14T18:25:37.843698+0000","last_fullsized":"2023-12-14T18:25:37.843698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:31:26.105116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.848178+0000","last_change":"2023-12-14T18:25:37.848178+0000","last_active":"2023-12-14T18:25:37.848178+0000","last_peered":"2023-12-14T18:25:37.848178+0000","last_clean":"2023-12-14T18:25:37.848178+0000","last_became_active":"2023-12-14T18:25:37.847717+0000","last_became_peered":"2023-12-14T18:25:37.847717+0000","last_unstale":"2023-12-14T18:25:37.848178+0000","last_undegraded":"2023-12-14T18:25:37.848178+0000","last_fullsized":"2023-12-14T18:25:37.848178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:42:37.509796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846056+0000","last_change":"2023-12-14T18:24:36.177484+0000","last_active":"2023-12-14T18:25:37.846056+0000","last_peered":"2023-12-14T18:25:37.846056+0000","last_clean":"2023-12-14T18:25:37.846056+0000","last_became_active":"2023-12-14T18:24:36.176592+0000","last_became_peered":"2023-12-14T18:24:36.176592+0000","last_unstale":"2023-12-14T18:25:37.846056+0000","last_undegraded":"2023-12-14T18:25:37.846056+0000","last_fullsized":"2023-12-14T18:25:37.846056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:19:41.645908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844073+0000","last_change":"2023-12-14T18:24:36.585687+0000","last_active":"2023-12-14T18:25:37.844073+0000","last_peered":"2023-12-14T18:25:37.844073+0000","last_clean":"2023-12-14T18:25:37.844073+0000","last_became_active":"2023-12-14T18:24:36.585360+0000","last_became_peered":"2023-12-14T18:24:36.585360+0000","last_unstale":"2023-12-14T18:25:37.844073+0000","last_undegraded":"2023-12-14T18:25:37.844073+0000","last_fullsized":"2023-12-14T18:25:37.844073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:32:14.790072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839687+0000","last_change":"2023-12-14T18:24:40.771747+0000","last_active":"2023-12-14T18:25:37.839687+0000","last_peered":"2023-12-14T18:25:37.839687+0000","last_clean":"2023-12-14T18:25:37.839687+0000","last_became_active":"2023-12-14T18:24:40.771310+0000","last_became_peered":"2023-12-14T18:24:40.771310+0000","last_unstale":"2023-12-14T18:25:37.839687+0000","last_undegraded":"2023-12-14T18:25:37.839687+0000","last_fullsized":"2023-12-14T18:25:37.839687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:16:09.218784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.853179+0000","last_change":"2023-12-14T18:25:37.853179+0000","last_active":"2023-12-14T18:25:37.853179+0000","last_peered":"2023-12-14T18:25:37.853179+0000","last_clean":"2023-12-14T18:25:37.853179+0000","last_became_active":"2023-12-14T18:25:37.848064+0000","last_became_peered":"2023-12-14T18:25:37.848064+0000","last_unstale":"2023-12-14T18:25:37.853179+0000","last_undegraded":"2023-12-14T18:25:37.853179+0000","last_fullsized":"2023-12-14T18:25:37.853179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:47:58.918839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840322+0000","last_change":"2023-12-14T18:24:36.594938+0000","last_active":"2023-12-14T18:25:37.840322+0000","last_peered":"2023-12-14T18:25:37.840322+0000","last_clean":"2023-12-14T18:25:37.840322+0000","last_became_active":"2023-12-14T18:24:36.594573+0000","last_became_peered":"2023-12-14T18:24:36.594573+0000","last_unstale":"2023-12-14T18:25:37.840322+0000","last_undegraded":"2023-12-14T18:25:37.840322+0000","last_fullsized":"2023-12-14T18:25:37.840322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:32:38.255137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840467+0000","last_change":"2023-12-14T18:24:40.601146+0000","last_active":"2023-12-14T18:25:37.840467+0000","last_peered":"2023-12-14T18:25:37.840467+0000","last_clean":"2023-12-14T18:25:37.840467+0000","last_became_active":"2023-12-14T18:24:40.600745+0000","last_became_peered":"2023-12-14T18:24:40.600745+0000","last_unstale":"2023-12-14T18:25:37.840467+0000","last_undegraded":"2023-12-14T18:25:37.840467+0000","last_fullsized":"2023-12-14T18:25:37.840467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:19:54.844673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844456+0000","last_change":"2023-12-14T18:24:36.175880+0000","last_active":"2023-12-14T18:25:37.844456+0000","last_peered":"2023-12-14T18:25:37.844456+0000","last_clean":"2023-12-14T18:25:37.844456+0000","last_became_active":"2023-12-14T18:24:36.175514+0000","last_became_peered":"2023-12-14T18:24:36.175514+0000","last_unstale":"2023-12-14T18:25:37.844456+0000","last_undegraded":"2023-12-14T18:25:37.844456+0000","last_fullsized":"2023-12-14T18:25:37.844456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:31:53.320299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052652+0000","last_change":"2023-12-14T18:24:36.577062+0000","last_active":"2023-12-14T18:25:38.052652+0000","last_peered":"2023-12-14T18:25:38.052652+0000","last_clean":"2023-12-14T18:25:38.052652+0000","last_became_active":"2023-12-14T18:24:36.576733+0000","last_became_peered":"2023-12-14T18:24:36.576733+0000","last_unstale":"2023-12-14T18:25:38.052652+0000","last_undegraded":"2023-12-14T18:25:38.052652+0000","last_fullsized":"2023-12-14T18:25:38.052652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:58.268670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844112+0000","last_change":"2023-12-14T18:24:40.772752+0000","last_active":"2023-12-14T18:25:37.844112+0000","last_peered":"2023-12-14T18:25:37.844112+0000","last_clean":"2023-12-14T18:25:37.844112+0000","last_became_active":"2023-12-14T18:24:40.772168+0000","last_became_peered":"2023-12-14T18:24:40.772168+0000","last_unstale":"2023-12-14T18:25:37.844112+0000","last_undegraded":"2023-12-14T18:25:37.844112+0000","last_fullsized":"2023-12-14T18:25:37.844112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:12:47.177023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840372+0000","last_change":"2023-12-14T18:24:36.178885+0000","last_active":"2023-12-14T18:25:37.840372+0000","last_peered":"2023-12-14T18:25:37.840372+0000","last_clean":"2023-12-14T18:25:37.840372+0000","last_became_active":"2023-12-14T18:24:36.178386+0000","last_became_peered":"2023-12-14T18:24:36.178386+0000","last_unstale":"2023-12-14T18:25:37.840372+0000","last_undegraded":"2023-12-14T18:25:37.840372+0000","last_fullsized":"2023-12-14T18:25:37.840372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:19:35.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842574+0000","last_change":"2023-12-14T18:24:36.590037+0000","last_active":"2023-12-14T18:25:37.842574+0000","last_peered":"2023-12-14T18:25:37.842574+0000","last_clean":"2023-12-14T18:25:37.842574+0000","last_became_active":"2023-12-14T18:24:36.589659+0000","last_became_peered":"2023-12-14T18:24:36.589659+0000","last_unstale":"2023-12-14T18:25:37.842574+0000","last_undegraded":"2023-12-14T18:25:37.842574+0000","last_fullsized":"2023-12-14T18:25:37.842574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:09:42.198821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.851489+0000","last_change":"2023-12-14T18:25:37.850532+0000","last_active":"2023-12-14T18:25:37.851489+0000","last_peered":"2023-12-14T18:25:37.851489+0000","last_clean":"2023-12-14T18:25:37.851489+0000","last_became_active":"2023-12-14T18:25:37.850082+0000","last_became_peered":"2023-12-14T18:25:37.850082+0000","last_unstale":"2023-12-14T18:25:37.851489+0000","last_undegraded":"2023-12-14T18:25:37.851489+0000","last_fullsized":"2023-12-14T18:25:37.851489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:24:47.887065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839807+0000","last_change":"2023-12-14T18:24:36.173350+0000","last_active":"2023-12-14T18:25:37.839807+0000","last_peered":"2023-12-14T18:25:37.839807+0000","last_clean":"2023-12-14T18:25:37.839807+0000","last_became_active":"2023-12-14T18:24:36.172840+0000","last_became_peered":"2023-12-14T18:24:36.172840+0000","last_unstale":"2023-12-14T18:25:37.839807+0000","last_undegraded":"2023-12-14T18:25:37.839807+0000","last_fullsized":"2023-12-14T18:25:37.839807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:03:03.230376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842563+0000","last_change":"2023-12-14T18:24:36.592104+0000","last_active":"2023-12-14T18:25:37.842563+0000","last_peered":"2023-12-14T18:25:37.842563+0000","last_clean":"2023-12-14T18:25:37.842563+0000","last_became_active":"2023-12-14T18:24:36.591568+0000","last_became_peered":"2023-12-14T18:24:36.591568+0000","last_unstale":"2023-12-14T18:25:37.842563+0000","last_undegraded":"2023-12-14T18:25:37.842563+0000","last_fullsized":"2023-12-14T18:25:37.842563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:38:19.772980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.059912+0000","last_change":"2023-12-14T18:25:38.059912+0000","last_active":"2023-12-14T18:25:38.059912+0000","last_peered":"2023-12-14T18:25:38.059912+0000","last_clean":"2023-12-14T18:25:38.059912+0000","last_became_active":"2023-12-14T18:25:38.054942+0000","last_became_peered":"2023-12-14T18:25:38.054942+0000","last_unstale":"2023-12-14T18:25:38.059912+0000","last_undegraded":"2023-12-14T18:25:38.059912+0000","last_fullsized":"2023-12-14T18:25:38.059912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:48.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.856268+0000","last_change":"2023-12-14T18:25:37.855817+0000","last_active":"2023-12-14T18:25:37.856268+0000","last_peered":"2023-12-14T18:25:37.856268+0000","last_clean":"2023-12-14T18:25:37.856268+0000","last_became_active":"2023-12-14T18:25:37.849192+0000","last_became_peered":"2023-12-14T18:25:37.849192+0000","last_unstale":"2023-12-14T18:25:37.856268+0000","last_undegraded":"2023-12-14T18:25:37.856268+0000","last_fullsized":"2023-12-14T18:25:37.856268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:23:11.825835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840095+0000","last_change":"2023-12-14T18:24:36.587777+0000","last_active":"2023-12-14T18:25:37.840095+0000","last_peered":"2023-12-14T18:25:37.840095+0000","last_clean":"2023-12-14T18:25:37.840095+0000","last_became_active":"2023-12-14T18:24:36.587317+0000","last_became_peered":"2023-12-14T18:24:36.587317+0000","last_unstale":"2023-12-14T18:25:37.840095+0000","last_undegraded":"2023-12-14T18:25:37.840095+0000","last_fullsized":"2023-12-14T18:25:37.840095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:22:21.777360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844980+0000","last_change":"2023-12-14T18:24:40.771913+0000","last_active":"2023-12-14T18:25:37.844980+0000","last_peered":"2023-12-14T18:25:37.844980+0000","last_clean":"2023-12-14T18:25:37.844980+0000","last_became_active":"2023-12-14T18:24:40.771425+0000","last_became_peered":"2023-12-14T18:24:40.771425+0000","last_unstale":"2023-12-14T18:25:37.844980+0000","last_undegraded":"2023-12-14T18:25:37.844980+0000","last_fullsized":"2023-12-14T18:25:37.844980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:20:55.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842694+0000","last_change":"2023-12-14T18:24:36.176801+0000","last_active":"2023-12-14T18:25:37.842694+0000","last_peered":"2023-12-14T18:25:37.842694+0000","last_clean":"2023-12-14T18:25:37.842694+0000","last_became_active":"2023-12-14T18:24:36.176001+0000","last_became_peered":"2023-12-14T18:24:36.176001+0000","last_unstale":"2023-12-14T18:25:37.842694+0000","last_undegraded":"2023-12-14T18:25:37.842694+0000","last_fullsized":"2023-12-14T18:25:37.842694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:31:08.426978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845322+0000","last_change":"2023-12-14T18:24:36.592898+0000","last_active":"2023-12-14T18:25:37.845322+0000","last_peered":"2023-12-14T18:25:37.845322+0000","last_clean":"2023-12-14T18:25:37.845322+0000","last_became_active":"2023-12-14T18:24:36.586650+0000","last_became_peered":"2023-12-14T18:24:36.586650+0000","last_unstale":"2023-12-14T18:25:37.845322+0000","last_undegraded":"2023-12-14T18:25:37.845322+0000","last_fullsized":"2023-12-14T18:25:37.845322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:35:59.247019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840411+0000","last_change":"2023-12-14T18:24:40.599181+0000","last_active":"2023-12-14T18:25:37.840411+0000","last_peered":"2023-12-14T18:25:37.840411+0000","last_clean":"2023-12-14T18:25:37.840411+0000","last_became_active":"2023-12-14T18:24:40.598785+0000","last_became_peered":"2023-12-14T18:24:40.598785+0000","last_unstale":"2023-12-14T18:25:37.840411+0000","last_undegraded":"2023-12-14T18:25:37.840411+0000","last_fullsized":"2023-12-14T18:25:37.840411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:55:55.159561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841700+0000","last_change":"2023-12-14T18:24:36.173644+0000","last_active":"2023-12-14T18:25:37.841700+0000","last_peered":"2023-12-14T18:25:37.841700+0000","last_clean":"2023-12-14T18:25:37.841700+0000","last_became_active":"2023-12-14T18:24:36.173059+0000","last_became_peered":"2023-12-14T18:24:36.173059+0000","last_unstale":"2023-12-14T18:25:37.841700+0000","last_undegraded":"2023-12-14T18:25:37.841700+0000","last_fullsized":"2023-12-14T18:25:37.841700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:16:27.909550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844525+0000","last_change":"2023-12-14T18:24:36.588761+0000","last_active":"2023-12-14T18:25:37.844525+0000","last_peered":"2023-12-14T18:25:37.844525+0000","last_clean":"2023-12-14T18:25:37.844525+0000","last_became_active":"2023-12-14T18:24:36.586021+0000","last_became_peered":"2023-12-14T18:24:36.586021+0000","last_unstale":"2023-12-14T18:25:37.844525+0000","last_undegraded":"2023-12-14T18:25:37.844525+0000","last_fullsized":"2023-12-14T18:25:37.844525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:45:41.729490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052349+0000","last_change":"2023-12-14T18:24:40.607286+0000","last_active":"2023-12-14T18:25:38.052349+0000","last_peered":"2023-12-14T18:25:38.052349+0000","last_clean":"2023-12-14T18:25:38.052349+0000","last_became_active":"2023-12-14T18:24:40.606925+0000","last_became_peered":"2023-12-14T18:24:40.606925+0000","last_unstale":"2023-12-14T18:25:38.052349+0000","last_undegraded":"2023-12-14T18:25:38.052349+0000","last_fullsized":"2023-12-14T18:25:38.052349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:26:27.152905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.847697+0000","last_change":"2023-12-14T18:24:36.174467+0000","last_active":"2023-12-14T18:25:37.847697+0000","last_peered":"2023-12-14T18:25:37.847697+0000","last_clean":"2023-12-14T18:25:37.847697+0000","last_became_active":"2023-12-14T18:24:36.173920+0000","last_became_peered":"2023-12-14T18:24:36.173920+0000","last_unstale":"2023-12-14T18:25:37.847697+0000","last_undegraded":"2023-12-14T18:25:37.847697+0000","last_fullsized":"2023-12-14T18:25:37.847697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:23:20.860799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846258+0000","last_change":"2023-12-14T18:24:36.597537+0000","last_active":"2023-12-14T18:25:37.846258+0000","last_peered":"2023-12-14T18:25:37.846258+0000","last_clean":"2023-12-14T18:25:37.846258+0000","last_became_active":"2023-12-14T18:24:36.594599+0000","last_became_peered":"2023-12-14T18:24:36.594599+0000","last_unstale":"2023-12-14T18:25:37.846258+0000","last_undegraded":"2023-12-14T18:25:37.846258+0000","last_fullsized":"2023-12-14T18:25:37.846258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:00:18.245087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844725+0000","last_change":"2023-12-14T18:24:40.597990+0000","last_active":"2023-12-14T18:25:37.844725+0000","last_peered":"2023-12-14T18:25:37.844725+0000","last_clean":"2023-12-14T18:25:37.844725+0000","last_became_active":"2023-12-14T18:24:40.597568+0000","last_became_peered":"2023-12-14T18:24:40.597568+0000","last_unstale":"2023-12-14T18:25:37.844725+0000","last_undegraded":"2023-12-14T18:25:37.844725+0000","last_fullsized":"2023-12-14T18:25:37.844725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:36:17.954844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844241+0000","last_change":"2023-12-14T18:24:36.173141+0000","last_active":"2023-12-14T18:25:37.844241+0000","last_peered":"2023-12-14T18:25:37.844241+0000","last_clean":"2023-12-14T18:25:37.844241+0000","last_became_active":"2023-12-14T18:24:36.172811+0000","last_became_peered":"2023-12-14T18:24:36.172811+0000","last_unstale":"2023-12-14T18:25:37.844241+0000","last_undegraded":"2023-12-14T18:25:37.844241+0000","last_fullsized":"2023-12-14T18:25:37.844241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:39:02.206044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844789+0000","last_change":"2023-12-14T18:24:36.587333+0000","last_active":"2023-12-14T18:25:37.844789+0000","last_peered":"2023-12-14T18:25:37.844789+0000","last_clean":"2023-12-14T18:25:37.844789+0000","last_became_active":"2023-12-14T18:24:36.586914+0000","last_became_peered":"2023-12-14T18:24:36.586914+0000","last_unstale":"2023-12-14T18:25:37.844789+0000","last_undegraded":"2023-12-14T18:25:37.844789+0000","last_fullsized":"2023-12-14T18:25:37.844789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:48:31.035981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842775+0000","last_change":"2023-12-14T18:24:36.591896+0000","last_active":"2023-12-14T18:25:37.842775+0000","last_peered":"2023-12-14T18:25:37.842775+0000","last_clean":"2023-12-14T18:25:37.842775+0000","last_became_active":"2023-12-14T18:24:36.591097+0000","last_became_peered":"2023-12-14T18:24:36.591097+0000","last_unstale":"2023-12-14T18:25:37.842775+0000","last_undegraded":"2023-12-14T18:25:37.842775+0000","last_fullsized":"2023-12-14T18:25:37.842775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:22:08.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841677+0000","last_change":"2023-12-14T18:24:36.581688+0000","last_active":"2023-12-14T18:25:37.841677+0000","last_peered":"2023-12-14T18:25:37.841677+0000","last_clean":"2023-12-14T18:25:37.841677+0000","last_became_active":"2023-12-14T18:24:36.580660+0000","last_became_peered":"2023-12-14T18:24:36.580660+0000","last_unstale":"2023-12-14T18:25:37.841677+0000","last_undegraded":"2023-12-14T18:25:37.841677+0000","last_fullsized":"2023-12-14T18:25:37.841677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:11:30.240797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840500+0000","last_change":"2023-12-14T18:24:36.583025+0000","last_active":"2023-12-14T18:25:37.840500+0000","last_peered":"2023-12-14T18:25:37.840500+0000","last_clean":"2023-12-14T18:25:37.840500+0000","last_became_active":"2023-12-14T18:24:36.582510+0000","last_became_peered":"2023-12-14T18:24:36.582510+0000","last_unstale":"2023-12-14T18:25:37.840500+0000","last_undegraded":"2023-12-14T18:25:37.840500+0000","last_fullsized":"2023-12-14T18:25:37.840500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:34:09.434901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052493+0000","last_change":"2023-12-14T18:24:36.579383+0000","last_active":"2023-12-14T18:25:38.052493+0000","last_peered":"2023-12-14T18:25:38.052493+0000","last_clean":"2023-12-14T18:25:38.052493+0000","last_became_active":"2023-12-14T18:24:36.579037+0000","last_became_peered":"2023-12-14T18:24:36.579037+0000","last_unstale":"2023-12-14T18:25:38.052493+0000","last_undegraded":"2023-12-14T18:25:38.052493+0000","last_fullsized":"2023-12-14T18:25:38.052493+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:20:41.833469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053594+0000","last_change":"2023-12-14T18:25:38.053594+0000","last_active":"2023-12-14T18:25:38.053594+0000","last_peered":"2023-12-14T18:25:38.053594+0000","last_clean":"2023-12-14T18:25:38.053594+0000","last_became_active":"2023-12-14T18:25:38.053317+0000","last_became_peered":"2023-12-14T18:25:38.053317+0000","last_unstale":"2023-12-14T18:25:38.053594+0000","last_undegraded":"2023-12-14T18:25:38.053594+0000","last_fullsized":"2023-12-14T18:25:38.053594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:19:29.086598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843497+0000","last_change":"2023-12-14T18:24:36.578613+0000","last_active":"2023-12-14T18:25:37.843497+0000","last_peered":"2023-12-14T18:25:37.843497+0000","last_clean":"2023-12-14T18:25:37.843497+0000","last_became_active":"2023-12-14T18:24:36.578287+0000","last_became_peered":"2023-12-14T18:24:36.578287+0000","last_unstale":"2023-12-14T18:25:37.843497+0000","last_undegraded":"2023-12-14T18:25:37.843497+0000","last_fullsized":"2023-12-14T18:25:37.843497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:09:07.746792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841403+0000","last_change":"2023-12-14T18:24:36.575078+0000","last_active":"2023-12-14T18:25:37.841403+0000","last_peered":"2023-12-14T18:25:37.841403+0000","last_clean":"2023-12-14T18:25:37.841403+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:25:37.841403+0000","last_undegraded":"2023-12-14T18:25:37.841403+0000","last_fullsized":"2023-12-14T18:25:37.841403+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:19:17.093417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840428+0000","last_change":"2023-12-14T18:24:36.572545+0000","last_active":"2023-12-14T18:25:37.840428+0000","last_peered":"2023-12-14T18:25:37.840428+0000","last_clean":"2023-12-14T18:25:37.840428+0000","last_became_active":"2023-12-14T18:24:36.572235+0000","last_became_peered":"2023-12-14T18:24:36.572235+0000","last_unstale":"2023-12-14T18:25:37.840428+0000","last_undegraded":"2023-12-14T18:25:37.840428+0000","last_fullsized":"2023-12-14T18:25:37.840428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:38:19.260777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845273+0000","last_change":"2023-12-14T18:24:36.588278+0000","last_active":"2023-12-14T18:25:37.845273+0000","last_peered":"2023-12-14T18:25:37.845273+0000","last_clean":"2023-12-14T18:25:37.845273+0000","last_became_active":"2023-12-14T18:24:36.587605+0000","last_became_peered":"2023-12-14T18:24:36.587605+0000","last_unstale":"2023-12-14T18:25:37.845273+0000","last_undegraded":"2023-12-14T18:25:37.845273+0000","last_fullsized":"2023-12-14T18:25:37.845273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:20:58.777764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839826+0000","last_change":"2023-12-14T18:24:36.575325+0000","last_active":"2023-12-14T18:25:37.839826+0000","last_peered":"2023-12-14T18:25:37.839826+0000","last_clean":"2023-12-14T18:25:37.839826+0000","last_became_active":"2023-12-14T18:24:36.575016+0000","last_became_peered":"2023-12-14T18:24:36.575016+0000","last_unstale":"2023-12-14T18:25:37.839826+0000","last_undegraded":"2023-12-14T18:25:37.839826+0000","last_fullsized":"2023-12-14T18:25:37.839826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:41:53.442267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840657+0000","last_change":"2023-12-14T18:24:36.585348+0000","last_active":"2023-12-14T18:25:37.840657+0000","last_peered":"2023-12-14T18:25:37.840657+0000","last_clean":"2023-12-14T18:25:37.840657+0000","last_became_active":"2023-12-14T18:24:36.584995+0000","last_became_peered":"2023-12-14T18:24:36.584995+0000","last_unstale":"2023-12-14T18:25:37.840657+0000","last_undegraded":"2023-12-14T18:25:37.840657+0000","last_fullsized":"2023-12-14T18:25:37.840657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:11:50.337217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843900+0000","last_change":"2023-12-14T18:24:36.583571+0000","last_active":"2023-12-14T18:25:37.843900+0000","last_peered":"2023-12-14T18:25:37.843900+0000","last_clean":"2023-12-14T18:25:37.843900+0000","last_became_active":"2023-12-14T18:24:36.583266+0000","last_became_peered":"2023-12-14T18:24:36.583266+0000","last_unstale":"2023-12-14T18:25:37.843900+0000","last_undegraded":"2023-12-14T18:25:37.843900+0000","last_fullsized":"2023-12-14T18:25:37.843900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:59:28.832769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843165+0000","last_change":"2023-12-14T18:24:36.579754+0000","last_active":"2023-12-14T18:25:37.843165+0000","last_peered":"2023-12-14T18:25:37.843165+0000","last_clean":"2023-12-14T18:25:37.843165+0000","last_became_active":"2023-12-14T18:24:36.579337+0000","last_became_peered":"2023-12-14T18:24:36.579337+0000","last_unstale":"2023-12-14T18:25:37.843165+0000","last_undegraded":"2023-12-14T18:25:37.843165+0000","last_fullsized":"2023-12-14T18:25:37.843165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:17:05.199500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212533,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64740,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":266,"num_read_kb":204,"num_write":153,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":964,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547223,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6112,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739040,"statfs":{"total":95995035648,"available":95988776960,"internally_reserved":0,"allocated":942080,"data_stored":477716,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.46999999999999997}]},{"osd":1,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.313},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.34399999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.41799999999999998},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.433},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.40100000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.497}]},{"osd":7,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.65200000000000002}]},{"osd":8,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.53300000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.57599999999999996}]}]},{"osd":10,"up_from":67,"seq":287762808859,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6572,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738580,"statfs":{"total":95995035648,"available":95988305920,"internally_reserved":0,"allocated":1085440,"data_stored":1129537,"data_compressed":53238,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":6.109,"5min":6.109,"15min":6.109},"last":0.57699999999999996},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":9.5180000000000007,"5min":9.5180000000000007,"15min":9.5180000000000007},"last":0.66300000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.6690000000000005,"5min":8.6690000000000005,"15min":8.6690000000000005},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.5},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":9.5459999999999994,"5min":9.5459999999999994,"15min":9.5459999999999994},"last":0.47699999999999998}]},{"osd":3,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":9.4380000000000006,"5min":9.4380000000000006,"15min":9.4380000000000006},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.56000000000000005}]},{"osd":4,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.9470000000000001,"5min":5.9470000000000001,"15min":5.9470000000000001},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":6.1470000000000002,"5min":6.1470000000000002,"15min":6.1470000000000002},"last":0.36599999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.59599999999999997}]},{"osd":6,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":9.4830000000000005,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.61499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.624}]},{"osd":9,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.69099999999999995}]},{"osd":11,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":6.0309999999999997,"5min":6.0309999999999997,"15min":6.0309999999999997},"last":0.70899999999999996}]}]},{"osd":9,"up_from":62,"seq":266287972382,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738832,"statfs":{"total":95995035648,"available":95988563968,"internally_reserved":0,"allocated":958464,"data_stored":477952,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42199999999999999,"15min":0.42999999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53500000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.27200000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.73999999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60999999999999999,"15min":0.62},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.83399999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.495}]},{"osd":2,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61199999999999999,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.47199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59199999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.879,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72999999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.45900000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80700000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.749},"min":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84799999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.441}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.81699999999999995,"5min":1.214,"15min":1.214},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.47199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79600000000000004,"5min":1.071,"15min":1.071},"last":0.59599999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54900000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55900000000000005}]}]},{"osd":8,"up_from":55,"seq":236223201314,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6444,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738708,"statfs":{"total":95995035648,"available":95988436992,"internally_reserved":0,"allocated":954368,"data_stored":477541,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.69699999999999995}]},{"osd":1,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.32500000000000001}]},{"osd":3,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.624}]},{"osd":4,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.67500000000000004},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.36599999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.63800000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.754,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.72599999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.70499999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.746}]}]},{"osd":7,"up_from":48,"seq":206158430245,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6556,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738596,"statfs":{"total":95995035648,"available":95988322304,"internally_reserved":0,"allocated":1003520,"data_stored":1073353,"data_compressed":53238,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.434}]},{"osd":1,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.32800000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.48599999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.53600000000000003,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.7830000000000004,"5min":5.7830000000000004,"15min":5.7830000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.39700000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.54800000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.52000000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77500000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.72599999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.58999999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82699999999999996,"5min":0.879,"15min":0.879},"last":0.60299999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.56399999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.629},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.71599999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.6220000000000001,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"min":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.8120000000000001,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.63900000000000001}]},{"osd":11,"last update":"Thu Dec 14 18:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.70599999999999996}]}]},{"osd":6,"up_from":43,"seq":184683593767,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6364,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738788,"statfs":{"total":95995035648,"available":95988518912,"internally_reserved":0,"allocated":937984,"data_stored":473128,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.26300000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.432},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.59699999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.39700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.54300000000000004}]},{"osd":2,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.45900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88800000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.77100000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81599999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.36599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.56200000000000006}]},{"osd":4,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86099999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.625}]},{"osd":5,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.39400000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.752}]},{"osd":7,"last update":"Thu Dec 14 18:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.624,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.68300000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.65200000000000002,"15min":0.65700000000000003},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79200000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.627,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79800000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.72699999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.64400000000000002}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.71599999999999997}]},{"osd":11,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574907,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7388,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737764,"statfs":{"total":95995035648,"available":95987470336,"internally_reserved":0,"allocated":1003520,"data_stored":533874,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.46200000000000002},"min":{"1min":0.36099999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.42699999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.42999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.41299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70599999999999996}]},{"osd":3,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47799999999999998,"15min":0.46500000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.5}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.316,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.40300000000000002},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77600000000000002,"5min":1.542,"15min":1.542},"last":0.46999999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53800000000000003,"15min":0.51900000000000002},"min":{"1min":0.442,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.52500000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83299999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.57499999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.501,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.78000000000000003}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52900000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.61099999999999999}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.75900000000000001}]},{"osd":11,"last update":"Thu Dec 14 18:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.749},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.82199999999999995}]}]},{"osd":0,"up_from":8,"seq":34359738430,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7220,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737932,"statfs":{"total":95995035648,"available":95987642368,"internally_reserved":0,"allocated":962560,"data_stored":474516,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.42799999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.52500000000000002,"15min":0.497},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54000000000000004,"15min":0.5},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54800000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:23:45 2023","interfaces":[{"interface":"back","average":{"1min":0.371,"5min":0.33400000000000002,"15min":0.32800000000000001},"min":{"1min":0.27300000000000002,"5min":0.19500000000000001,"15min":0.19500000000000001},"max":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"last":0.378},{"interface":"front","average":{"1min":0.38,"5min":0.35899999999999999,"15min":0.35499999999999998},"min":{"1min":0.26700000000000002,"5min":0.17499999999999999,"15min":0.17499999999999999},"max":{"1min":0.54200000000000004,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.46300000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.38,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51200000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71799999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.50600000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.51300000000000001,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.40400000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.57799999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:24:40 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.46100000000000002,"15min":0.46700000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61699999999999999,"5min":0.71399999999999997,"15min":0.71399999999999997},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47999999999999998,"15min":0.48199999999999998},"min":{"1min":0.318,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.53900000000000003}]},{"osd":7,"last update":"Thu Dec 14 18:23:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.63800000000000001,"15min":0.63800000000000001},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.61099999999999999,"15min":0.61099999999999999},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.66200000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.63300000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.63900000000000001,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":7.2290000000000001,"5min":7.2290000000000001,"15min":7.2290000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.67300000000000004,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.37,"15min":0.37},"max":{"1min":7.3390000000000004,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.64100000000000001}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.70399999999999996},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.623}]},{"osd":11,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.65200000000000002}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7532,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737620,"statfs":{"total":95995035648,"available":95987322880,"internally_reserved":0,"allocated":1019904,"data_stored":534602,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47599999999999998,"15min":0.442},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.45500000000000002},"min":{"1min":0.377,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.54600000000000004,"15min":0.48699999999999999},"min":{"1min":0.51100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.55900000000000005,"15min":0.501},"min":{"1min":0.45600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.46600000000000003},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76100000000000001,"5min":1.179,"15min":1.179},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Dec 14 18:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.748}]},{"osd":7,"last update":"Thu Dec 14 18:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53000000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73099999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.625},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.70599999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71999999999999997}]},{"osd":11,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.80900000000000005}]}]},{"osd":3,"up_from":24,"seq":103079215155,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6696,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738456,"statfs":{"total":95995035648,"available":95988178944,"internally_reserved":0,"allocated":950272,"data_stored":473322,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.32800000000000001},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.248,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.35799999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.497,"15min":0.47499999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.55300000000000005}]},{"osd":2,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":1.214,"15min":1.214},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.45300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57099999999999995}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64100000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.45200000000000001}]},{"osd":6,"last update":"Thu Dec 14 18:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.28499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.73699999999999999,"5min":0.755,"15min":0.755},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.437,"5min":0.51000000000000001,"15min":0.53000000000000003},"min":{"1min":0.34899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64500000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.439}]},{"osd":7,"last update":"Thu Dec 14 18:24:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.628},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.61299999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79900000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.63700000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.54400000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.66700000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.64600000000000002}]}]},{"osd":4,"up_from":31,"seq":133143986224,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6512,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738640,"statfs":{"total":95995035648,"available":95988367360,"internally_reserved":0,"allocated":958464,"data_stored":478694,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48799999999999999,"15min":0.48099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.70199999999999996}]},{"osd":2,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.48399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.502},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Dec 14 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.45000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.76500000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.57599999999999996}]},{"osd":6,"last update":"Thu Dec 14 18:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.67100000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.437,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72199999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.72199999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.78100000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.503,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.77000000000000002}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.51900000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.84499999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.86799999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.81000000000000005}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.83699999999999997},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.79600000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.84999999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822700,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6696,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738456,"statfs":{"total":95995035648,"available":95988178944,"internally_reserved":0,"allocated":1015808,"data_stored":1069320,"data_compressed":53238,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.626,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.42399999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.501,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.62}]},{"osd":2,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.254,"15min":0.254},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.32800000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.628,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.35299999999999998}]},{"osd":3,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.68000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.63100000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.60999999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":0.997},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.50600000000000001}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.56000000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82299999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.57499999999999996}]},{"osd":9,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.58599999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.61799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.66600000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.71999999999999997,"15min":0.71999999999999997},"min":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.93700000000000006,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.69399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:25:45.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:44 smithi097 ceph-mon[43486]: pgmap v260: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 255 B/s wr, 6 op/s; 135 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:46.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:45 smithi160 ceph-mon[43840]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:46.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:45 smithi093 ceph-mon[42070]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:46.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:45 smithi097 ceph-mon[43486]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:46 smithi160 ceph-mon[43840]: pgmap v261: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s; 119 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:47.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:46 smithi093 ceph-mon[42070]: pgmap v261: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s; 119 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:47.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:46 smithi097 ceph-mon[43486]: pgmap v261: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 KiB/s rd, 3 op/s; 119 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:48.311 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph pg dump --format=json 2023-12-14T18:25:48.641 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:49.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:49 smithi093 ceph-mon[42070]: pgmap v262: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 107 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:49.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:49 smithi097 ceph-mon[43486]: pgmap v262: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 107 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:49.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:49 smithi160 ceph-mon[43840]: pgmap v262: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s; 107 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:49.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.620+0000 7f87050bd700 1 -- 172.21.15.93:0/1371484936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100560 msgr2=0x7f8700100930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:49.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.620+0000 7f87050bd700 1 --2- 172.21.15.93:0/1371484936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100560 0x7f8700100930 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f86f40099c0 tx=0x7f86f4009cd0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.620+0000 7f87050bd700 1 -- 172.21.15.93:0/1371484936 shutdown_connections 2023-12-14T18:25:49.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.620+0000 7f87050bd700 1 --2- 172.21.15.93:0/1371484936 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100560 0x7f8700100930 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.620+0000 7f87050bd700 1 --2- 172.21.15.93:0/1371484936 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8700105600 0x7f870010d180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.620+0000 7f87050bd700 1 --2- 172.21.15.93:0/1371484936 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8700100f00 0x7f8700104ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.620+0000 7f87050bd700 1 -- 172.21.15.93:0/1371484936 >> 172.21.15.93:0/1371484936 conn(0x7f87000fbde0 msgr2=0x7f87000fe1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:49.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.620+0000 7f87050bd700 1 -- 172.21.15.93:0/1371484936 shutdown_connections 2023-12-14T18:25:49.623 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.620+0000 7f87050bd700 1 -- 172.21.15.93:0/1371484936 wait complete. 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f87050bd700 1 Processor -- start 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f87050bd700 1 -- start start 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f87050bd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8700100560 0x7f8700196420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f87050bd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100f00 0x7f8700196960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f87050bd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8700105600 0x7f870019add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f87050bd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870019b3e0 con 0x7f8700100f00 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f87050bd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870019b550 con 0x7f8700105600 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f87050bd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f870019b710 con 0x7f8700100560 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86fe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100f00 0x7f8700196960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86fe59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100f00 0x7f8700196960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:35566/0 (socket says 172.21.15.93:35566) 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86fe59c700 1 -- 172.21.15.93:0/2997589338 learned_addr learned my addr 172.21.15.93:0/2997589338 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86fed9d700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8700100560 0x7f8700196420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86ff59e700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8700105600 0x7f870019add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86fe59c700 1 -- 172.21.15.93:0/2997589338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8700100560 msgr2=0x7f8700196420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:49.624 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86fe59c700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8700100560 0x7f8700196420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86fe59c700 1 -- 172.21.15.93:0/2997589338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8700105600 msgr2=0x7f870019add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86fe59c700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8700105600 0x7f870019add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86fe59c700 1 -- 172.21.15.93:0/2997589338 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86f40095f0 con 0x7f8700100f00 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.621+0000 7f86ff59e700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8700105600 0x7f870019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.622+0000 7f86fe59c700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100f00 0x7f8700196960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f86f000d980 tx=0x7f86f000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.622+0000 7f86e7fff700 1 -- 172.21.15.93:0/2997589338 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f0009930 con 0x7f8700100f00 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.622+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8700110390 con 0x7f8700100f00 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.622+0000 7f86e7fff700 1 -- 172.21.15.93:0/2997589338 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f86f0010460 con 0x7f8700100f00 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.622+0000 7f86e7fff700 1 -- 172.21.15.93:0/2997589338 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86f000f620 con 0x7f8700100f00 2023-12-14T18:25:49.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.622+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8700110200 con 0x7f8700100f00 2023-12-14T18:25:49.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.623+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8700110520 con 0x7f8700100f00 2023-12-14T18:25:49.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.624+0000 7f86e7fff700 1 -- 172.21.15.93:0/2997589338 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f86f00108d0 con 0x7f8700100f00 2023-12-14T18:25:49.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.624+0000 7f86e7fff700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f86e80a11e0 0x7f86e80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:49.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.625+0000 7f86fed9d700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f86e80a11e0 0x7f86e80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:49.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.625+0000 7f86e7fff700 1 -- 172.21.15.93:0/2997589338 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f86f00cc860 con 0x7f8700100f00 2023-12-14T18:25:49.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.625+0000 7f86fed9d700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f86e80a11e0 0x7f86e80a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f86f4006010 tx=0x7f86f4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:49.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.629+0000 7f86e7fff700 1 -- 172.21.15.93:0/2997589338 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f86f0095b40 con 0x7f8700100f00 2023-12-14T18:25:49.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.816+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f87000fcf70 con 0x7f86e80a11e0 2023-12-14T18:25:49.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.827+0000 7f86e7fff700 1 -- 172.21.15.93:0/2997589338 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+427489 (secure 0 0 0) 0x7f87000fcf70 con 0x7f86e80a11e0 2023-12-14T18:25:49.834 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:49.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f86e80a11e0 msgr2=0x7f86e80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f86e80a11e0 0x7f86e80a3690 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f86f4006010 tx=0x7f86f4009f10 comp rx=0 tx=0).stop 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100f00 msgr2=0x7f8700196960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100f00 0x7f8700196960 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f86f000d980 tx=0x7f86f000dc90 comp rx=0 tx=0).stop 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 shutdown_connections 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f86e80a11e0 0x7f86e80a3690 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8700105600 0x7f870019add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8700100560 0x7f8700196420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 --2- 172.21.15.93:0/2997589338 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8700100f00 0x7f8700196960 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 >> 172.21.15.93:0/2997589338 conn(0x7f87000fbde0 msgr2=0x7f87000fc860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 shutdown_connections 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:49.831+0000 7f87050bd700 1 -- 172.21.15.93:0/2997589338 wait complete. 2023-12-14T18:25:49.838 INFO:teuthology.orchestra.run.smithi093.stderr:dumped all 2023-12-14T18:25:50.204 INFO:teuthology.orchestra.run.smithi093.stdout:{"pg_ready":true,"pg_map":{"version":262,"stamp":"2023-12-14T18:25:47.963607+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":655354,"num_objects":64,"num_object_clones":0,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":0,"num_read":432,"num_read_kb":338,"num_write":336,"num_write_kb":3049,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":1181823,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":421,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":80412,"kb_used_data":11516,"kb_used_omap":0,"kb_used_meta":68800,"kb_avail":1124861412,"statfs":{"total":1151940427776,"available":1151858085888,"internally_reserved":0,"allocated":11792384,"data_stored":7673555,"data_compressed":317601,"data_compressed_allocated":430080,"data_compressed_original":2359296,"omap_allocated":0,"internal_metadata":70451200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":60,"num_read_kb":30,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":1087,"num_keys_recovered":7,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.125025"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844182+0000","last_change":"2023-12-14T18:24:36.587133+0000","last_active":"2023-12-14T18:25:37.844182+0000","last_peered":"2023-12-14T18:25:37.844182+0000","last_clean":"2023-12-14T18:25:37.844182+0000","last_became_active":"2023-12-14T18:24:36.586222+0000","last_became_peered":"2023-12-14T18:24:36.586222+0000","last_unstale":"2023-12-14T18:25:37.844182+0000","last_undegraded":"2023-12-14T18:25:37.844182+0000","last_fullsized":"2023-12-14T18:25:37.844182+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:35:18.142196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052570+0000","last_change":"2023-12-14T18:24:36.589027+0000","last_active":"2023-12-14T18:25:38.052570+0000","last_peered":"2023-12-14T18:25:38.052570+0000","last_clean":"2023-12-14T18:25:38.052570+0000","last_became_active":"2023-12-14T18:24:36.588785+0000","last_became_peered":"2023-12-14T18:24:36.588785+0000","last_unstale":"2023-12-14T18:25:38.052570+0000","last_undegraded":"2023-12-14T18:25:38.052570+0000","last_fullsized":"2023-12-14T18:25:38.052570+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:10:31.188239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052726+0000","last_change":"2023-12-14T18:24:36.584015+0000","last_active":"2023-12-14T18:25:38.052726+0000","last_peered":"2023-12-14T18:25:38.052726+0000","last_clean":"2023-12-14T18:25:38.052726+0000","last_became_active":"2023-12-14T18:24:36.583735+0000","last_became_peered":"2023-12-14T18:24:36.583735+0000","last_unstale":"2023-12-14T18:25:38.052726+0000","last_undegraded":"2023-12-14T18:25:38.052726+0000","last_fullsized":"2023-12-14T18:25:38.052726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:35:38.893225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844897+0000","last_change":"2023-12-14T18:24:36.584553+0000","last_active":"2023-12-14T18:25:37.844897+0000","last_peered":"2023-12-14T18:25:37.844897+0000","last_clean":"2023-12-14T18:25:37.844897+0000","last_became_active":"2023-12-14T18:24:36.583827+0000","last_became_peered":"2023-12-14T18:24:36.583827+0000","last_unstale":"2023-12-14T18:25:37.844897+0000","last_undegraded":"2023-12-14T18:25:37.844897+0000","last_fullsized":"2023-12-14T18:25:37.844897+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:25:09.201735+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842858+0000","last_change":"2023-12-14T18:24:36.578128+0000","last_active":"2023-12-14T18:25:37.842858+0000","last_peered":"2023-12-14T18:25:37.842858+0000","last_clean":"2023-12-14T18:25:37.842858+0000","last_became_active":"2023-12-14T18:24:36.577784+0000","last_became_peered":"2023-12-14T18:24:36.577784+0000","last_unstale":"2023-12-14T18:25:37.842858+0000","last_undegraded":"2023-12-14T18:25:37.842858+0000","last_fullsized":"2023-12-14T18:25:37.842858+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:43:10.548891+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053740+0000","last_change":"2023-12-14T18:24:36.588666+0000","last_active":"2023-12-14T18:25:38.053740+0000","last_peered":"2023-12-14T18:25:38.053740+0000","last_clean":"2023-12-14T18:25:38.053740+0000","last_became_active":"2023-12-14T18:24:36.588291+0000","last_became_peered":"2023-12-14T18:24:36.588291+0000","last_unstale":"2023-12-14T18:25:38.053740+0000","last_undegraded":"2023-12-14T18:25:38.053740+0000","last_fullsized":"2023-12-14T18:25:38.053740+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:17:14.408821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844209+0000","last_change":"2023-12-14T18:24:36.586637+0000","last_active":"2023-12-14T18:25:37.844209+0000","last_peered":"2023-12-14T18:25:37.844209+0000","last_clean":"2023-12-14T18:25:37.844209+0000","last_became_active":"2023-12-14T18:24:36.583055+0000","last_became_peered":"2023-12-14T18:24:36.583055+0000","last_unstale":"2023-12-14T18:25:37.844209+0000","last_undegraded":"2023-12-14T18:25:37.844209+0000","last_fullsized":"2023-12-14T18:25:37.844209+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:18:17.900284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839806+0000","last_change":"2023-12-14T18:24:36.580206+0000","last_active":"2023-12-14T18:25:37.839806+0000","last_peered":"2023-12-14T18:25:37.839806+0000","last_clean":"2023-12-14T18:25:37.839806+0000","last_became_active":"2023-12-14T18:24:36.579904+0000","last_became_peered":"2023-12-14T18:24:36.579904+0000","last_unstale":"2023-12-14T18:25:37.839806+0000","last_undegraded":"2023-12-14T18:25:37.839806+0000","last_fullsized":"2023-12-14T18:25:37.839806+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:40:36.607985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841073+0000","last_change":"2023-12-14T18:24:36.589890+0000","last_active":"2023-12-14T18:25:37.841073+0000","last_peered":"2023-12-14T18:25:37.841073+0000","last_clean":"2023-12-14T18:25:37.841073+0000","last_became_active":"2023-12-14T18:24:36.588999+0000","last_became_peered":"2023-12-14T18:24:36.588999+0000","last_unstale":"2023-12-14T18:25:37.841073+0000","last_undegraded":"2023-12-14T18:25:37.841073+0000","last_fullsized":"2023-12-14T18:25:37.841073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:12:20.169411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.054309+0000","last_change":"2023-12-14T18:24:36.589565+0000","last_active":"2023-12-14T18:25:38.054309+0000","last_peered":"2023-12-14T18:25:38.054309+0000","last_clean":"2023-12-14T18:25:38.054309+0000","last_became_active":"2023-12-14T18:24:36.588822+0000","last_became_peered":"2023-12-14T18:24:36.588822+0000","last_unstale":"2023-12-14T18:25:38.054309+0000","last_undegraded":"2023-12-14T18:25:38.054309+0000","last_fullsized":"2023-12-14T18:25:38.054309+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:03:50.712722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844272+0000","last_change":"2023-12-14T18:24:36.593208+0000","last_active":"2023-12-14T18:25:37.844272+0000","last_peered":"2023-12-14T18:25:37.844272+0000","last_clean":"2023-12-14T18:25:37.844272+0000","last_became_active":"2023-12-14T18:24:36.586394+0000","last_became_peered":"2023-12-14T18:24:36.586394+0000","last_unstale":"2023-12-14T18:25:37.844272+0000","last_undegraded":"2023-12-14T18:25:37.844272+0000","last_fullsized":"2023-12-14T18:25:37.844272+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:50:22.984010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":11,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.055045+0000","last_change":"2023-12-14T18:25:38.054725+0000","last_active":"2023-12-14T18:25:38.055045+0000","last_peered":"2023-12-14T18:25:38.055045+0000","last_clean":"2023-12-14T18:25:38.055045+0000","last_became_active":"2023-12-14T18:25:38.054505+0000","last_became_peered":"2023-12-14T18:25:38.054505+0000","last_unstale":"2023-12-14T18:25:38.055045+0000","last_undegraded":"2023-12-14T18:25:38.055045+0000","last_fullsized":"2023-12-14T18:25:38.055045+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:29:35.537082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,10,8],"acting":[9,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843790+0000","last_change":"2023-12-14T18:24:36.584971+0000","last_active":"2023-12-14T18:25:37.843790+0000","last_peered":"2023-12-14T18:25:37.843790+0000","last_clean":"2023-12-14T18:25:37.843790+0000","last_became_active":"2023-12-14T18:24:36.584695+0000","last_became_peered":"2023-12-14T18:24:36.584695+0000","last_unstale":"2023-12-14T18:25:37.843790+0000","last_undegraded":"2023-12-14T18:25:37.843790+0000","last_fullsized":"2023-12-14T18:25:37.843790+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:18:22.200476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052278+0000","last_change":"2023-12-14T18:24:36.589692+0000","last_active":"2023-12-14T18:25:38.052278+0000","last_peered":"2023-12-14T18:25:38.052278+0000","last_clean":"2023-12-14T18:25:38.052278+0000","last_became_active":"2023-12-14T18:24:36.589415+0000","last_became_peered":"2023-12-14T18:24:36.589415+0000","last_unstale":"2023-12-14T18:25:38.052278+0000","last_undegraded":"2023-12-14T18:25:38.052278+0000","last_fullsized":"2023-12-14T18:25:38.052278+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:34:32.470917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052263+0000","last_change":"2023-12-14T18:24:36.586911+0000","last_active":"2023-12-14T18:25:38.052263+0000","last_peered":"2023-12-14T18:25:38.052263+0000","last_clean":"2023-12-14T18:25:38.052263+0000","last_became_active":"2023-12-14T18:24:36.586600+0000","last_became_peered":"2023-12-14T18:24:36.586600+0000","last_unstale":"2023-12-14T18:25:38.052263+0000","last_undegraded":"2023-12-14T18:25:38.052263+0000","last_fullsized":"2023-12-14T18:25:38.052263+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:05:03.670077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840245+0000","last_change":"2023-12-14T18:24:36.584879+0000","last_active":"2023-12-14T18:25:37.840245+0000","last_peered":"2023-12-14T18:25:37.840245+0000","last_clean":"2023-12-14T18:25:37.840245+0000","last_became_active":"2023-12-14T18:24:36.584566+0000","last_became_peered":"2023-12-14T18:24:36.584566+0000","last_unstale":"2023-12-14T18:25:37.840245+0000","last_undegraded":"2023-12-14T18:25:37.840245+0000","last_fullsized":"2023-12-14T18:25:37.840245+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:28:45.762443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843862+0000","last_change":"2023-12-14T18:24:36.587420+0000","last_active":"2023-12-14T18:25:37.843862+0000","last_peered":"2023-12-14T18:25:37.843862+0000","last_clean":"2023-12-14T18:25:37.843862+0000","last_became_active":"2023-12-14T18:24:36.586732+0000","last_became_peered":"2023-12-14T18:24:36.586732+0000","last_unstale":"2023-12-14T18:25:37.843862+0000","last_undegraded":"2023-12-14T18:25:37.843862+0000","last_fullsized":"2023-12-14T18:25:37.843862+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:05:08.359976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842943+0000","last_change":"2023-12-14T18:24:36.583412+0000","last_active":"2023-12-14T18:25:37.842943+0000","last_peered":"2023-12-14T18:25:37.842943+0000","last_clean":"2023-12-14T18:25:37.842943+0000","last_became_active":"2023-12-14T18:24:36.583022+0000","last_became_peered":"2023-12-14T18:24:36.583022+0000","last_unstale":"2023-12-14T18:25:37.842943+0000","last_undegraded":"2023-12-14T18:25:37.842943+0000","last_fullsized":"2023-12-14T18:25:37.842943+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:07:57.550138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841957+0000","last_change":"2023-12-14T18:24:36.575232+0000","last_active":"2023-12-14T18:25:37.841957+0000","last_peered":"2023-12-14T18:25:37.841957+0000","last_clean":"2023-12-14T18:25:37.841957+0000","last_became_active":"2023-12-14T18:24:36.574573+0000","last_became_peered":"2023-12-14T18:24:36.574573+0000","last_unstale":"2023-12-14T18:25:37.841957+0000","last_undegraded":"2023-12-14T18:25:37.841957+0000","last_fullsized":"2023-12-14T18:25:37.841957+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:16:28.555112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840943+0000","last_change":"2023-12-14T18:24:40.605077+0000","last_active":"2023-12-14T18:25:37.840943+0000","last_peered":"2023-12-14T18:25:37.840943+0000","last_clean":"2023-12-14T18:25:37.840943+0000","last_became_active":"2023-12-14T18:24:40.604646+0000","last_became_peered":"2023-12-14T18:24:40.604646+0000","last_unstale":"2023-12-14T18:25:37.840943+0000","last_undegraded":"2023-12-14T18:25:37.840943+0000","last_fullsized":"2023-12-14T18:25:37.840943+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:59:21.706575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'9","reported_seq":36,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842596+0000","last_change":"2023-12-14T18:24:36.177449+0000","last_active":"2023-12-14T18:25:37.842596+0000","last_peered":"2023-12-14T18:25:37.842596+0000","last_clean":"2023-12-14T18:25:37.842596+0000","last_became_active":"2023-12-14T18:24:36.177114+0000","last_became_peered":"2023-12-14T18:24:36.177114+0000","last_unstale":"2023-12-14T18:25:37.842596+0000","last_undegraded":"2023-12-14T18:25:37.842596+0000","last_fullsized":"2023-12-14T18:25:37.842596+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:33:20.038616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840363+0000","last_change":"2023-12-14T18:24:36.574204+0000","last_active":"2023-12-14T18:25:37.840363+0000","last_peered":"2023-12-14T18:25:37.840363+0000","last_clean":"2023-12-14T18:25:37.840363+0000","last_became_active":"2023-12-14T18:24:36.573923+0000","last_became_peered":"2023-12-14T18:24:36.573923+0000","last_unstale":"2023-12-14T18:25:37.840363+0000","last_undegraded":"2023-12-14T18:25:37.840363+0000","last_fullsized":"2023-12-14T18:25:37.840363+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:16:19.375764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843661+0000","last_change":"2023-12-14T18:24:40.600254+0000","last_active":"2023-12-14T18:25:37.843661+0000","last_peered":"2023-12-14T18:25:37.843661+0000","last_clean":"2023-12-14T18:25:37.843661+0000","last_became_active":"2023-12-14T18:24:40.599894+0000","last_became_peered":"2023-12-14T18:24:40.599894+0000","last_unstale":"2023-12-14T18:25:37.843661+0000","last_undegraded":"2023-12-14T18:25:37.843661+0000","last_fullsized":"2023-12-14T18:25:37.843661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:53:10.654777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840496+0000","last_change":"2023-12-14T18:24:36.170109+0000","last_active":"2023-12-14T18:25:37.840496+0000","last_peered":"2023-12-14T18:25:37.840496+0000","last_clean":"2023-12-14T18:25:37.840496+0000","last_became_active":"2023-12-14T18:24:36.169629+0000","last_became_peered":"2023-12-14T18:24:36.169629+0000","last_unstale":"2023-12-14T18:25:37.840496+0000","last_undegraded":"2023-12-14T18:25:37.840496+0000","last_fullsized":"2023-12-14T18:25:37.840496+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:15:23.380992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052504+0000","last_change":"2023-12-14T18:24:36.582757+0000","last_active":"2023-12-14T18:25:38.052504+0000","last_peered":"2023-12-14T18:25:38.052504+0000","last_clean":"2023-12-14T18:25:38.052504+0000","last_became_active":"2023-12-14T18:24:36.582457+0000","last_became_peered":"2023-12-14T18:24:36.582457+0000","last_unstale":"2023-12-14T18:25:38.052504+0000","last_undegraded":"2023-12-14T18:25:38.052504+0000","last_fullsized":"2023-12-14T18:25:38.052504+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:35:13.993694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845045+0000","last_change":"2023-12-14T18:24:40.599695+0000","last_active":"2023-12-14T18:25:37.845045+0000","last_peered":"2023-12-14T18:25:37.845045+0000","last_clean":"2023-12-14T18:25:37.845045+0000","last_became_active":"2023-12-14T18:24:40.599239+0000","last_became_peered":"2023-12-14T18:24:40.599239+0000","last_unstale":"2023-12-14T18:25:37.845045+0000","last_undegraded":"2023-12-14T18:25:37.845045+0000","last_fullsized":"2023-12-14T18:25:37.845045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:54:19.445064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846330+0000","last_change":"2023-12-14T18:24:36.177946+0000","last_active":"2023-12-14T18:25:37.846330+0000","last_peered":"2023-12-14T18:25:37.846330+0000","last_clean":"2023-12-14T18:25:37.846330+0000","last_became_active":"2023-12-14T18:24:36.177480+0000","last_became_peered":"2023-12-14T18:24:36.177480+0000","last_unstale":"2023-12-14T18:25:37.846330+0000","last_undegraded":"2023-12-14T18:25:37.846330+0000","last_fullsized":"2023-12-14T18:25:37.846330+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:58:29.278977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840508+0000","last_change":"2023-12-14T18:24:36.587951+0000","last_active":"2023-12-14T18:25:37.840508+0000","last_peered":"2023-12-14T18:25:37.840508+0000","last_clean":"2023-12-14T18:25:37.840508+0000","last_became_active":"2023-12-14T18:24:36.587539+0000","last_became_peered":"2023-12-14T18:24:36.587539+0000","last_unstale":"2023-12-14T18:25:37.840508+0000","last_undegraded":"2023-12-14T18:25:37.840508+0000","last_fullsized":"2023-12-14T18:25:37.840508+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:18:40.567638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843868+0000","last_change":"2023-12-14T18:24:40.600008+0000","last_active":"2023-12-14T18:25:37.843868+0000","last_peered":"2023-12-14T18:25:37.843868+0000","last_clean":"2023-12-14T18:25:37.843868+0000","last_became_active":"2023-12-14T18:24:40.599596+0000","last_became_peered":"2023-12-14T18:24:40.599596+0000","last_unstale":"2023-12-14T18:25:37.843868+0000","last_undegraded":"2023-12-14T18:25:37.843868+0000","last_fullsized":"2023-12-14T18:25:37.843868+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:06:21.571807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"79'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845392+0000","last_change":"2023-12-14T18:24:36.176569+0000","last_active":"2023-12-14T18:25:37.845392+0000","last_peered":"2023-12-14T18:25:37.845392+0000","last_clean":"2023-12-14T18:25:37.845392+0000","last_became_active":"2023-12-14T18:24:36.176043+0000","last_became_peered":"2023-12-14T18:24:36.176043+0000","last_unstale":"2023-12-14T18:25:37.845392+0000","last_undegraded":"2023-12-14T18:25:37.845392+0000","last_fullsized":"2023-12-14T18:25:37.845392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:37:45.325365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.847594+0000","last_change":"2023-12-14T18:24:36.593761+0000","last_active":"2023-12-14T18:25:37.847594+0000","last_peered":"2023-12-14T18:25:37.847594+0000","last_clean":"2023-12-14T18:25:37.847594+0000","last_became_active":"2023-12-14T18:24:36.593452+0000","last_became_peered":"2023-12-14T18:24:36.593452+0000","last_unstale":"2023-12-14T18:25:37.847594+0000","last_undegraded":"2023-12-14T18:25:37.847594+0000","last_fullsized":"2023-12-14T18:25:37.847594+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:05:51.179026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841661+0000","last_change":"2023-12-14T18:24:40.598064+0000","last_active":"2023-12-14T18:25:37.841661+0000","last_peered":"2023-12-14T18:25:37.841661+0000","last_clean":"2023-12-14T18:25:37.841661+0000","last_became_active":"2023-12-14T18:24:40.597672+0000","last_became_peered":"2023-12-14T18:24:40.597672+0000","last_unstale":"2023-12-14T18:25:37.841661+0000","last_undegraded":"2023-12-14T18:25:37.841661+0000","last_fullsized":"2023-12-14T18:25:37.841661+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:19:47.256702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'6","reported_seq":94,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:45.862869+0000","last_change":"2023-12-14T18:25:37.854647+0000","last_active":"2023-12-14T18:25:45.862869+0000","last_peered":"2023-12-14T18:25:45.862869+0000","last_clean":"2023-12-14T18:25:45.862869+0000","last_became_active":"2023-12-14T18:25:37.848559+0000","last_became_peered":"2023-12-14T18:25:37.848559+0000","last_unstale":"2023-12-14T18:25:45.862869+0000","last_undegraded":"2023-12-14T18:25:45.862869+0000","last_fullsized":"2023-12-14T18:25:45.862869+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:03.962836+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":964,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":964,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843816+0000","last_change":"2023-12-14T18:24:36.583687+0000","last_active":"2023-12-14T18:25:37.843816+0000","last_peered":"2023-12-14T18:25:37.843816+0000","last_clean":"2023-12-14T18:25:37.843816+0000","last_became_active":"2023-12-14T18:24:36.583047+0000","last_became_peered":"2023-12-14T18:24:36.583047+0000","last_unstale":"2023-12-14T18:25:37.843816+0000","last_undegraded":"2023-12-14T18:25:37.843816+0000","last_fullsized":"2023-12-14T18:25:37.843816+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:59:28.910079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841353+0000","last_change":"2023-12-14T18:24:40.605217+0000","last_active":"2023-12-14T18:25:37.841353+0000","last_peered":"2023-12-14T18:25:37.841353+0000","last_clean":"2023-12-14T18:25:37.841353+0000","last_became_active":"2023-12-14T18:24:40.604807+0000","last_became_peered":"2023-12-14T18:24:40.604807+0000","last_unstale":"2023-12-14T18:25:37.841353+0000","last_undegraded":"2023-12-14T18:25:37.841353+0000","last_fullsized":"2023-12-14T18:25:37.841353+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:00:15.863206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053667+0000","last_change":"2023-12-14T18:24:36.174875+0000","last_active":"2023-12-14T18:25:38.053667+0000","last_peered":"2023-12-14T18:25:38.053667+0000","last_clean":"2023-12-14T18:25:38.053667+0000","last_became_active":"2023-12-14T18:24:36.173984+0000","last_became_peered":"2023-12-14T18:24:36.173984+0000","last_unstale":"2023-12-14T18:25:38.053667+0000","last_undegraded":"2023-12-14T18:25:38.053667+0000","last_fullsized":"2023-12-14T18:25:38.053667+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:09:45.465327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846977+0000","last_change":"2023-12-14T18:24:36.580563+0000","last_active":"2023-12-14T18:25:37.846977+0000","last_peered":"2023-12-14T18:25:37.846977+0000","last_clean":"2023-12-14T18:25:37.846977+0000","last_became_active":"2023-12-14T18:24:36.580196+0000","last_became_peered":"2023-12-14T18:24:36.580196+0000","last_unstale":"2023-12-14T18:25:37.846977+0000","last_undegraded":"2023-12-14T18:25:37.846977+0000","last_fullsized":"2023-12-14T18:25:37.846977+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:32:02.350091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843700+0000","last_change":"2023-12-14T18:24:40.601223+0000","last_active":"2023-12-14T18:25:37.843700+0000","last_peered":"2023-12-14T18:25:37.843700+0000","last_clean":"2023-12-14T18:25:37.843700+0000","last_became_active":"2023-12-14T18:24:40.600795+0000","last_became_peered":"2023-12-14T18:24:40.600795+0000","last_unstale":"2023-12-14T18:25:37.843700+0000","last_undegraded":"2023-12-14T18:25:37.843700+0000","last_fullsized":"2023-12-14T18:25:37.843700+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:34:31.442312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843493+0000","last_change":"2023-12-14T18:24:36.173600+0000","last_active":"2023-12-14T18:25:37.843493+0000","last_peered":"2023-12-14T18:25:37.843493+0000","last_clean":"2023-12-14T18:25:37.843493+0000","last_became_active":"2023-12-14T18:24:36.173272+0000","last_became_peered":"2023-12-14T18:24:36.173272+0000","last_unstale":"2023-12-14T18:25:37.843493+0000","last_undegraded":"2023-12-14T18:25:37.843493+0000","last_fullsized":"2023-12-14T18:25:37.843493+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:36:35.523553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839726+0000","last_change":"2023-12-14T18:24:36.575060+0000","last_active":"2023-12-14T18:25:37.839726+0000","last_peered":"2023-12-14T18:25:37.839726+0000","last_clean":"2023-12-14T18:25:37.839726+0000","last_became_active":"2023-12-14T18:24:36.574749+0000","last_became_peered":"2023-12-14T18:24:36.574749+0000","last_unstale":"2023-12-14T18:25:37.839726+0000","last_undegraded":"2023-12-14T18:25:37.839726+0000","last_fullsized":"2023-12-14T18:25:37.839726+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:23:49.611073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052543+0000","last_change":"2023-12-14T18:24:40.605431+0000","last_active":"2023-12-14T18:25:38.052543+0000","last_peered":"2023-12-14T18:25:38.052543+0000","last_clean":"2023-12-14T18:25:38.052543+0000","last_became_active":"2023-12-14T18:24:40.604951+0000","last_became_peered":"2023-12-14T18:24:40.604951+0000","last_unstale":"2023-12-14T18:25:38.052543+0000","last_undegraded":"2023-12-14T18:25:38.052543+0000","last_fullsized":"2023-12-14T18:25:38.052543+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:52:55.859566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"84'1","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845146+0000","last_change":"2023-12-14T18:24:36.175493+0000","last_active":"2023-12-14T18:25:37.845146+0000","last_peered":"2023-12-14T18:25:37.845146+0000","last_clean":"2023-12-14T18:25:37.845146+0000","last_became_active":"2023-12-14T18:24:36.175205+0000","last_became_peered":"2023-12-14T18:24:36.175205+0000","last_unstale":"2023-12-14T18:25:37.845146+0000","last_undegraded":"2023-12-14T18:25:37.845146+0000","last_fullsized":"2023-12-14T18:25:37.845146+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:15:17.424259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842088+0000","last_change":"2023-12-14T18:24:36.587218+0000","last_active":"2023-12-14T18:25:37.842088+0000","last_peered":"2023-12-14T18:25:37.842088+0000","last_clean":"2023-12-14T18:25:37.842088+0000","last_became_active":"2023-12-14T18:24:36.586910+0000","last_became_peered":"2023-12-14T18:24:36.586910+0000","last_unstale":"2023-12-14T18:25:37.842088+0000","last_undegraded":"2023-12-14T18:25:37.842088+0000","last_fullsized":"2023-12-14T18:25:37.842088+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:51:27.244227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839960+0000","last_change":"2023-12-14T18:24:40.598409+0000","last_active":"2023-12-14T18:25:37.839960+0000","last_peered":"2023-12-14T18:25:37.839960+0000","last_clean":"2023-12-14T18:25:37.839960+0000","last_became_active":"2023-12-14T18:24:40.598011+0000","last_became_peered":"2023-12-14T18:24:40.598011+0000","last_unstale":"2023-12-14T18:25:37.839960+0000","last_undegraded":"2023-12-14T18:25:37.839960+0000","last_fullsized":"2023-12-14T18:25:37.839960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:02:12.191033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"84'3","reported_seq":75,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:44.860885+0000","last_change":"2023-12-14T18:24:36.172277+0000","last_active":"2023-12-14T18:25:44.860885+0000","last_peered":"2023-12-14T18:25:44.860885+0000","last_clean":"2023-12-14T18:25:44.860885+0000","last_became_active":"2023-12-14T18:24:36.171907+0000","last_became_peered":"2023-12-14T18:24:36.171907+0000","last_unstale":"2023-12-14T18:25:44.860885+0000","last_undegraded":"2023-12-14T18:25:44.860885+0000","last_fullsized":"2023-12-14T18:25:44.860885+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:51:05.472295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":46,"num_read_kb":46,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840975+0000","last_change":"2023-12-14T18:24:36.584783+0000","last_active":"2023-12-14T18:25:37.840975+0000","last_peered":"2023-12-14T18:25:37.840975+0000","last_clean":"2023-12-14T18:25:37.840975+0000","last_became_active":"2023-12-14T18:24:36.584424+0000","last_became_peered":"2023-12-14T18:24:36.584424+0000","last_unstale":"2023-12-14T18:25:37.840975+0000","last_undegraded":"2023-12-14T18:25:37.840975+0000","last_fullsized":"2023-12-14T18:25:37.840975+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:44:21.204303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841959+0000","last_change":"2023-12-14T18:24:40.603176+0000","last_active":"2023-12-14T18:25:37.841959+0000","last_peered":"2023-12-14T18:25:37.841959+0000","last_clean":"2023-12-14T18:25:37.841959+0000","last_became_active":"2023-12-14T18:24:40.602868+0000","last_became_peered":"2023-12-14T18:24:40.602868+0000","last_unstale":"2023-12-14T18:25:37.841959+0000","last_undegraded":"2023-12-14T18:25:37.841959+0000","last_fullsized":"2023-12-14T18:25:37.841959+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:15:57.475854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844517+0000","last_change":"2023-12-14T18:24:36.177638+0000","last_active":"2023-12-14T18:25:37.844517+0000","last_peered":"2023-12-14T18:25:37.844517+0000","last_clean":"2023-12-14T18:25:37.844517+0000","last_became_active":"2023-12-14T18:24:36.177335+0000","last_became_peered":"2023-12-14T18:24:36.177335+0000","last_unstale":"2023-12-14T18:25:37.844517+0000","last_undegraded":"2023-12-14T18:25:37.844517+0000","last_fullsized":"2023-12-14T18:25:37.844517+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:29:24.204595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052948+0000","last_change":"2023-12-14T18:24:36.577508+0000","last_active":"2023-12-14T18:25:38.052948+0000","last_peered":"2023-12-14T18:25:38.052948+0000","last_clean":"2023-12-14T18:25:38.052948+0000","last_became_active":"2023-12-14T18:24:36.577196+0000","last_became_peered":"2023-12-14T18:24:36.577196+0000","last_unstale":"2023-12-14T18:25:38.052948+0000","last_undegraded":"2023-12-14T18:25:38.052948+0000","last_fullsized":"2023-12-14T18:25:38.052948+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:55:24.227563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845217+0000","last_change":"2023-12-14T18:24:40.594816+0000","last_active":"2023-12-14T18:25:37.845217+0000","last_peered":"2023-12-14T18:25:37.845217+0000","last_clean":"2023-12-14T18:25:37.845217+0000","last_became_active":"2023-12-14T18:24:40.594371+0000","last_became_peered":"2023-12-14T18:24:40.594371+0000","last_unstale":"2023-12-14T18:25:37.845217+0000","last_undegraded":"2023-12-14T18:25:37.845217+0000","last_fullsized":"2023-12-14T18:25:37.845217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:02:50.352263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'4","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.847468+0000","last_change":"2023-12-14T18:24:36.174238+0000","last_active":"2023-12-14T18:25:37.847468+0000","last_peered":"2023-12-14T18:25:37.847468+0000","last_clean":"2023-12-14T18:25:37.847468+0000","last_became_active":"2023-12-14T18:24:36.173373+0000","last_became_peered":"2023-12-14T18:24:36.173373+0000","last_unstale":"2023-12-14T18:25:37.847468+0000","last_undegraded":"2023-12-14T18:25:37.847468+0000","last_fullsized":"2023-12-14T18:25:37.847468+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:48:08.262361+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840699+0000","last_change":"2023-12-14T18:24:36.588589+0000","last_active":"2023-12-14T18:25:37.840699+0000","last_peered":"2023-12-14T18:25:37.840699+0000","last_clean":"2023-12-14T18:25:37.840699+0000","last_became_active":"2023-12-14T18:24:36.588269+0000","last_became_peered":"2023-12-14T18:24:36.588269+0000","last_unstale":"2023-12-14T18:25:37.840699+0000","last_undegraded":"2023-12-14T18:25:37.840699+0000","last_fullsized":"2023-12-14T18:25:37.840699+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:55:55.150514+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842244+0000","last_change":"2023-12-14T18:24:40.601333+0000","last_active":"2023-12-14T18:25:37.842244+0000","last_peered":"2023-12-14T18:25:37.842244+0000","last_clean":"2023-12-14T18:25:37.842244+0000","last_became_active":"2023-12-14T18:24:40.600966+0000","last_became_peered":"2023-12-14T18:24:40.600966+0000","last_unstale":"2023-12-14T18:25:37.842244+0000","last_undegraded":"2023-12-14T18:25:37.842244+0000","last_fullsized":"2023-12-14T18:25:37.842244+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:30:57.057783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"84'31","reported_seq":157,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:45.860073+0000","last_change":"2023-12-14T18:24:36.177299+0000","last_active":"2023-12-14T18:25:45.860073+0000","last_peered":"2023-12-14T18:25:45.860073+0000","last_clean":"2023-12-14T18:25:45.860073+0000","last_became_active":"2023-12-14T18:24:36.176971+0000","last_became_peered":"2023-12-14T18:24:36.176971+0000","last_unstale":"2023-12-14T18:25:45.860073+0000","last_undegraded":"2023-12-14T18:25:45.860073+0000","last_fullsized":"2023-12-14T18:25:45.860073+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:09:55.682617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60836,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":100,"num_read_kb":109,"num_write":32,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846054+0000","last_change":"2023-12-14T18:24:36.586315+0000","last_active":"2023-12-14T18:25:37.846054+0000","last_peered":"2023-12-14T18:25:37.846054+0000","last_clean":"2023-12-14T18:25:37.846054+0000","last_became_active":"2023-12-14T18:24:36.586000+0000","last_became_peered":"2023-12-14T18:24:36.586000+0000","last_unstale":"2023-12-14T18:25:37.846054+0000","last_undegraded":"2023-12-14T18:25:37.846054+0000","last_fullsized":"2023-12-14T18:25:37.846054+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:55:32.691488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.847864+0000","last_change":"2023-12-14T18:24:40.596973+0000","last_active":"2023-12-14T18:25:37.847864+0000","last_peered":"2023-12-14T18:25:37.847864+0000","last_clean":"2023-12-14T18:25:37.847864+0000","last_became_active":"2023-12-14T18:24:40.596355+0000","last_became_peered":"2023-12-14T18:24:40.596355+0000","last_unstale":"2023-12-14T18:25:37.847864+0000","last_undegraded":"2023-12-14T18:25:37.847864+0000","last_fullsized":"2023-12-14T18:25:37.847864+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:05:47.138776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"84'6","reported_seq":33,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840994+0000","last_change":"2023-12-14T18:24:36.175287+0000","last_active":"2023-12-14T18:25:37.840994+0000","last_peered":"2023-12-14T18:25:37.840994+0000","last_clean":"2023-12-14T18:25:37.840994+0000","last_became_active":"2023-12-14T18:24:36.174952+0000","last_became_peered":"2023-12-14T18:24:36.174952+0000","last_unstale":"2023-12-14T18:25:37.840994+0000","last_undegraded":"2023-12-14T18:25:37.840994+0000","last_fullsized":"2023-12-14T18:25:37.840994+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:56:55.084380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":9,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843323+0000","last_change":"2023-12-14T18:24:36.582314+0000","last_active":"2023-12-14T18:25:37.843323+0000","last_peered":"2023-12-14T18:25:37.843323+0000","last_clean":"2023-12-14T18:25:37.843323+0000","last_became_active":"2023-12-14T18:24:36.581963+0000","last_became_peered":"2023-12-14T18:24:36.581963+0000","last_unstale":"2023-12-14T18:25:37.843323+0000","last_undegraded":"2023-12-14T18:25:37.843323+0000","last_fullsized":"2023-12-14T18:25:37.843323+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:01:58.910449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843110+0000","last_change":"2023-12-14T18:24:40.599979+0000","last_active":"2023-12-14T18:25:37.843110+0000","last_peered":"2023-12-14T18:25:37.843110+0000","last_clean":"2023-12-14T18:25:37.843110+0000","last_became_active":"2023-12-14T18:24:40.599258+0000","last_became_peered":"2023-12-14T18:24:40.599258+0000","last_unstale":"2023-12-14T18:25:37.843110+0000","last_undegraded":"2023-12-14T18:25:37.843110+0000","last_fullsized":"2023-12-14T18:25:37.843110+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:16:48.809775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846703+0000","last_change":"2023-12-14T18:24:36.175768+0000","last_active":"2023-12-14T18:25:37.846703+0000","last_peered":"2023-12-14T18:25:37.846703+0000","last_clean":"2023-12-14T18:25:37.846703+0000","last_became_active":"2023-12-14T18:24:36.174973+0000","last_became_peered":"2023-12-14T18:24:36.174973+0000","last_unstale":"2023-12-14T18:25:37.846703+0000","last_undegraded":"2023-12-14T18:25:37.846703+0000","last_fullsized":"2023-12-14T18:25:37.846703+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:41:10.229183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844716+0000","last_change":"2023-12-14T18:24:36.591856+0000","last_active":"2023-12-14T18:25:37.844716+0000","last_peered":"2023-12-14T18:25:37.844716+0000","last_clean":"2023-12-14T18:25:37.844716+0000","last_became_active":"2023-12-14T18:24:36.591554+0000","last_became_peered":"2023-12-14T18:24:36.591554+0000","last_unstale":"2023-12-14T18:25:37.844716+0000","last_undegraded":"2023-12-14T18:25:37.844716+0000","last_fullsized":"2023-12-14T18:25:37.844716+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:07:38.543120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842256+0000","last_change":"2023-12-14T18:24:40.603794+0000","last_active":"2023-12-14T18:25:37.842256+0000","last_peered":"2023-12-14T18:25:37.842256+0000","last_clean":"2023-12-14T18:25:37.842256+0000","last_became_active":"2023-12-14T18:24:40.603311+0000","last_became_peered":"2023-12-14T18:24:40.603311+0000","last_unstale":"2023-12-14T18:25:37.842256+0000","last_undegraded":"2023-12-14T18:25:37.842256+0000","last_fullsized":"2023-12-14T18:25:37.842256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:27:38.742031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"84'3","reported_seq":75,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:43.848492+0000","last_change":"2023-12-14T18:24:36.177411+0000","last_active":"2023-12-14T18:25:43.848492+0000","last_peered":"2023-12-14T18:25:43.848492+0000","last_clean":"2023-12-14T18:25:43.848492+0000","last_became_active":"2023-12-14T18:24:36.177047+0000","last_became_peered":"2023-12-14T18:24:36.177047+0000","last_unstale":"2023-12-14T18:25:43.848492+0000","last_undegraded":"2023-12-14T18:25:43.848492+0000","last_fullsized":"2023-12-14T18:25:43.848492+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:25:34.315245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":540,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":47,"num_read_kb":3,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.054628+0000","last_change":"2023-12-14T18:25:38.054628+0000","last_active":"2023-12-14T18:25:38.054628+0000","last_peered":"2023-12-14T18:25:38.054628+0000","last_clean":"2023-12-14T18:25:38.054628+0000","last_became_active":"2023-12-14T18:25:38.053719+0000","last_became_peered":"2023-12-14T18:25:38.053719+0000","last_unstale":"2023-12-14T18:25:38.054628+0000","last_undegraded":"2023-12-14T18:25:38.054628+0000","last_fullsized":"2023-12-14T18:25:38.054628+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:19:00.666432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,10],"acting":[7,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844240+0000","last_change":"2023-12-14T18:24:40.607042+0000","last_active":"2023-12-14T18:25:37.844240+0000","last_peered":"2023-12-14T18:25:37.844240+0000","last_clean":"2023-12-14T18:25:37.844240+0000","last_became_active":"2023-12-14T18:24:40.606563+0000","last_became_peered":"2023-12-14T18:24:40.606563+0000","last_unstale":"2023-12-14T18:25:37.844240+0000","last_undegraded":"2023-12-14T18:25:37.844240+0000","last_fullsized":"2023-12-14T18:25:37.844240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:17:48.751456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'15","reported_seq":42,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842804+0000","last_change":"2023-12-14T18:24:36.171994+0000","last_active":"2023-12-14T18:25:37.842804+0000","last_peered":"2023-12-14T18:25:37.842804+0000","last_clean":"2023-12-14T18:25:37.842804+0000","last_became_active":"2023-12-14T18:24:36.171661+0000","last_became_peered":"2023-12-14T18:24:36.171661+0000","last_unstale":"2023-12-14T18:25:37.842804+0000","last_undegraded":"2023-12-14T18:25:37.842804+0000","last_fullsized":"2023-12-14T18:25:37.842804+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:00:31.916889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845012+0000","last_change":"2023-12-14T18:24:36.577818+0000","last_active":"2023-12-14T18:25:37.845012+0000","last_peered":"2023-12-14T18:25:37.845012+0000","last_clean":"2023-12-14T18:25:37.845012+0000","last_became_active":"2023-12-14T18:24:36.577356+0000","last_became_peered":"2023-12-14T18:24:36.577356+0000","last_unstale":"2023-12-14T18:25:37.845012+0000","last_undegraded":"2023-12-14T18:25:37.845012+0000","last_fullsized":"2023-12-14T18:25:37.845012+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:27:12.642108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844848+0000","last_change":"2023-12-14T18:24:40.601995+0000","last_active":"2023-12-14T18:25:37.844848+0000","last_peered":"2023-12-14T18:25:37.844848+0000","last_clean":"2023-12-14T18:25:37.844848+0000","last_became_active":"2023-12-14T18:24:40.601571+0000","last_became_peered":"2023-12-14T18:24:40.601571+0000","last_unstale":"2023-12-14T18:25:37.844848+0000","last_undegraded":"2023-12-14T18:25:37.844848+0000","last_fullsized":"2023-12-14T18:25:37.844848+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:37:19.308982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'2","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846519+0000","last_change":"2023-12-14T18:24:36.168661+0000","last_active":"2023-12-14T18:25:37.846519+0000","last_peered":"2023-12-14T18:25:37.846519+0000","last_clean":"2023-12-14T18:25:37.846519+0000","last_became_active":"2023-12-14T18:24:36.168299+0000","last_became_peered":"2023-12-14T18:24:36.168299+0000","last_unstale":"2023-12-14T18:25:37.846519+0000","last_undegraded":"2023-12-14T18:25:37.846519+0000","last_fullsized":"2023-12-14T18:25:37.846519+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:13:54.195843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843080+0000","last_change":"2023-12-14T18:24:36.578417+0000","last_active":"2023-12-14T18:25:37.843080+0000","last_peered":"2023-12-14T18:25:37.843080+0000","last_clean":"2023-12-14T18:25:37.843080+0000","last_became_active":"2023-12-14T18:24:36.578080+0000","last_became_peered":"2023-12-14T18:24:36.578080+0000","last_unstale":"2023-12-14T18:25:37.843080+0000","last_undegraded":"2023-12-14T18:25:37.843080+0000","last_fullsized":"2023-12-14T18:25:37.843080+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:55:52.265641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.055456+0000","last_change":"2023-12-14T18:25:38.055456+0000","last_active":"2023-12-14T18:25:38.055456+0000","last_peered":"2023-12-14T18:25:38.055456+0000","last_clean":"2023-12-14T18:25:38.055456+0000","last_became_active":"2023-12-14T18:25:38.055025+0000","last_became_peered":"2023-12-14T18:25:38.055025+0000","last_unstale":"2023-12-14T18:25:38.055456+0000","last_undegraded":"2023-12-14T18:25:38.055456+0000","last_fullsized":"2023-12-14T18:25:38.055456+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:18:47.589447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842458+0000","last_change":"2023-12-14T18:24:36.170489+0000","last_active":"2023-12-14T18:25:37.842458+0000","last_peered":"2023-12-14T18:25:37.842458+0000","last_clean":"2023-12-14T18:25:37.842458+0000","last_became_active":"2023-12-14T18:24:36.170161+0000","last_became_peered":"2023-12-14T18:24:36.170161+0000","last_unstale":"2023-12-14T18:25:37.842458+0000","last_undegraded":"2023-12-14T18:25:37.842458+0000","last_fullsized":"2023-12-14T18:25:37.842458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:49:25.035687+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053967+0000","last_change":"2023-12-14T18:24:36.591094+0000","last_active":"2023-12-14T18:25:38.053967+0000","last_peered":"2023-12-14T18:25:38.053967+0000","last_clean":"2023-12-14T18:25:38.053967+0000","last_became_active":"2023-12-14T18:24:36.590779+0000","last_became_peered":"2023-12-14T18:24:36.590779+0000","last_unstale":"2023-12-14T18:25:38.053967+0000","last_undegraded":"2023-12-14T18:25:38.053967+0000","last_fullsized":"2023-12-14T18:25:38.053967+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:58:59.945071+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843917+0000","last_change":"2023-12-14T18:24:40.604125+0000","last_active":"2023-12-14T18:25:37.843917+0000","last_peered":"2023-12-14T18:25:37.843917+0000","last_clean":"2023-12-14T18:25:37.843917+0000","last_became_active":"2023-12-14T18:24:40.603242+0000","last_became_peered":"2023-12-14T18:24:40.603242+0000","last_unstale":"2023-12-14T18:25:37.843917+0000","last_undegraded":"2023-12-14T18:25:37.843917+0000","last_fullsized":"2023-12-14T18:25:37.843917+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:48:08.760578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"84'1","reported_seq":27,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845182+0000","last_change":"2023-12-14T18:24:36.177567+0000","last_active":"2023-12-14T18:25:37.845182+0000","last_peered":"2023-12-14T18:25:37.845182+0000","last_clean":"2023-12-14T18:25:37.845182+0000","last_became_active":"2023-12-14T18:24:36.176949+0000","last_became_peered":"2023-12-14T18:24:36.176949+0000","last_unstale":"2023-12-14T18:25:37.845182+0000","last_undegraded":"2023-12-14T18:25:37.845182+0000","last_fullsized":"2023-12-14T18:25:37.845182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:41:12.145991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053915+0000","last_change":"2023-12-14T18:24:36.594224+0000","last_active":"2023-12-14T18:25:38.053915+0000","last_peered":"2023-12-14T18:25:38.053915+0000","last_clean":"2023-12-14T18:25:38.053915+0000","last_became_active":"2023-12-14T18:24:36.593859+0000","last_became_peered":"2023-12-14T18:24:36.593859+0000","last_unstale":"2023-12-14T18:25:38.053915+0000","last_undegraded":"2023-12-14T18:25:38.053915+0000","last_fullsized":"2023-12-14T18:25:38.053915+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:12:34.937171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844213+0000","last_change":"2023-12-14T18:24:40.771814+0000","last_active":"2023-12-14T18:25:37.844213+0000","last_peered":"2023-12-14T18:25:37.844213+0000","last_clean":"2023-12-14T18:25:37.844213+0000","last_became_active":"2023-12-14T18:24:40.771387+0000","last_became_peered":"2023-12-14T18:24:40.771387+0000","last_unstale":"2023-12-14T18:25:37.844213+0000","last_undegraded":"2023-12-14T18:25:37.844213+0000","last_fullsized":"2023-12-14T18:25:37.844213+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:50:15.946615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845399+0000","last_change":"2023-12-14T18:24:36.588826+0000","last_active":"2023-12-14T18:25:37.845399+0000","last_peered":"2023-12-14T18:25:37.845399+0000","last_clean":"2023-12-14T18:25:37.845399+0000","last_became_active":"2023-12-14T18:24:36.588320+0000","last_became_peered":"2023-12-14T18:24:36.588320+0000","last_unstale":"2023-12-14T18:25:37.845399+0000","last_undegraded":"2023-12-14T18:25:37.845399+0000","last_fullsized":"2023-12-14T18:25:37.845399+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:41:32.298321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"84'4","reported_seq":75,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:44.849307+0000","last_change":"2023-12-14T18:24:36.173995+0000","last_active":"2023-12-14T18:25:44.849307+0000","last_peered":"2023-12-14T18:25:44.849307+0000","last_clean":"2023-12-14T18:25:44.849307+0000","last_became_active":"2023-12-14T18:24:36.173381+0000","last_became_peered":"2023-12-14T18:24:36.173381+0000","last_unstale":"2023-12-14T18:25:44.849307+0000","last_undegraded":"2023-12-14T18:25:44.849307+0000","last_fullsized":"2023-12-14T18:25:44.849307+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:05:11.493177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"84'1","reported_seq":18,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844039+0000","last_change":"2023-12-14T18:24:40.605099+0000","last_active":"2023-12-14T18:25:37.844039+0000","last_peered":"2023-12-14T18:25:37.844039+0000","last_clean":"2023-12-14T18:25:37.844039+0000","last_became_active":"2023-12-14T18:24:40.604773+0000","last_became_peered":"2023-12-14T18:24:40.604773+0000","last_unstale":"2023-12-14T18:25:37.844039+0000","last_undegraded":"2023-12-14T18:25:37.844039+0000","last_fullsized":"2023-12-14T18:25:37.844039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:01:03.932113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839914+0000","last_change":"2023-12-14T18:24:36.574993+0000","last_active":"2023-12-14T18:25:37.839914+0000","last_peered":"2023-12-14T18:25:37.839914+0000","last_clean":"2023-12-14T18:25:37.839914+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:25:37.839914+0000","last_undegraded":"2023-12-14T18:25:37.839914+0000","last_fullsized":"2023-12-14T18:25:37.839914+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:32:30.973673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"84'3","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845214+0000","last_change":"2023-12-14T18:24:36.170061+0000","last_active":"2023-12-14T18:25:37.845214+0000","last_peered":"2023-12-14T18:25:37.845214+0000","last_clean":"2023-12-14T18:25:37.845214+0000","last_became_active":"2023-12-14T18:24:36.169652+0000","last_became_peered":"2023-12-14T18:24:36.169652+0000","last_unstale":"2023-12-14T18:25:37.845214+0000","last_undegraded":"2023-12-14T18:25:37.845214+0000","last_fullsized":"2023-12-14T18:25:37.845214+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:46:50.366062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":4,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'137","reported_seq":348,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843576+0000","last_change":"2023-12-14T18:23:49.076644+0000","last_active":"2023-12-14T18:25:37.843576+0000","last_peered":"2023-12-14T18:25:37.843576+0000","last_clean":"2023-12-14T18:25:37.843576+0000","last_became_active":"2023-12-14T18:23:49.076263+0000","last_became_peered":"2023-12-14T18:23:49.076263+0000","last_unstale":"2023-12-14T18:25:37.843576+0000","last_undegraded":"2023-12-14T18:25:37.843576+0000","last_fullsized":"2023-12-14T18:25:37.843576+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:21:24.715131+0000","last_clean_scrub_stamp":"2023-12-14T18:21:24.715131+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:26:53.561034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.848093+0000","last_change":"2023-12-14T18:24:40.772015+0000","last_active":"2023-12-14T18:25:37.848093+0000","last_peered":"2023-12-14T18:25:37.848093+0000","last_clean":"2023-12-14T18:25:37.848093+0000","last_became_active":"2023-12-14T18:24:40.771497+0000","last_became_peered":"2023-12-14T18:24:40.771497+0000","last_unstale":"2023-12-14T18:25:37.848093+0000","last_undegraded":"2023-12-14T18:25:37.848093+0000","last_fullsized":"2023-12-14T18:25:37.848093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:50:50.964766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842573+0000","last_change":"2023-12-14T18:24:36.584402+0000","last_active":"2023-12-14T18:25:37.842573+0000","last_peered":"2023-12-14T18:25:37.842573+0000","last_clean":"2023-12-14T18:25:37.842573+0000","last_became_active":"2023-12-14T18:24:36.584177+0000","last_became_peered":"2023-12-14T18:24:36.584177+0000","last_unstale":"2023-12-14T18:25:37.842573+0000","last_undegraded":"2023-12-14T18:25:37.842573+0000","last_fullsized":"2023-12-14T18:25:37.842573+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:44:22.049474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843698+0000","last_change":"2023-12-14T18:24:36.175017+0000","last_active":"2023-12-14T18:25:37.843698+0000","last_peered":"2023-12-14T18:25:37.843698+0000","last_clean":"2023-12-14T18:25:37.843698+0000","last_became_active":"2023-12-14T18:24:36.174639+0000","last_became_peered":"2023-12-14T18:24:36.174639+0000","last_unstale":"2023-12-14T18:25:37.843698+0000","last_undegraded":"2023-12-14T18:25:37.843698+0000","last_fullsized":"2023-12-14T18:25:37.843698+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:31:26.105116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.848178+0000","last_change":"2023-12-14T18:25:37.848178+0000","last_active":"2023-12-14T18:25:37.848178+0000","last_peered":"2023-12-14T18:25:37.848178+0000","last_clean":"2023-12-14T18:25:37.848178+0000","last_became_active":"2023-12-14T18:25:37.847717+0000","last_became_peered":"2023-12-14T18:25:37.847717+0000","last_unstale":"2023-12-14T18:25:37.848178+0000","last_undegraded":"2023-12-14T18:25:37.848178+0000","last_fullsized":"2023-12-14T18:25:37.848178+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:42:37.509796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"84'1","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846056+0000","last_change":"2023-12-14T18:24:36.177484+0000","last_active":"2023-12-14T18:25:37.846056+0000","last_peered":"2023-12-14T18:25:37.846056+0000","last_clean":"2023-12-14T18:25:37.846056+0000","last_became_active":"2023-12-14T18:24:36.176592+0000","last_became_peered":"2023-12-14T18:24:36.176592+0000","last_unstale":"2023-12-14T18:25:37.846056+0000","last_undegraded":"2023-12-14T18:25:37.846056+0000","last_fullsized":"2023-12-14T18:25:37.846056+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:19:41.645908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844073+0000","last_change":"2023-12-14T18:24:36.585687+0000","last_active":"2023-12-14T18:25:37.844073+0000","last_peered":"2023-12-14T18:25:37.844073+0000","last_clean":"2023-12-14T18:25:37.844073+0000","last_became_active":"2023-12-14T18:24:36.585360+0000","last_became_peered":"2023-12-14T18:24:36.585360+0000","last_unstale":"2023-12-14T18:25:37.844073+0000","last_undegraded":"2023-12-14T18:25:37.844073+0000","last_fullsized":"2023-12-14T18:25:37.844073+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:32:14.790072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839687+0000","last_change":"2023-12-14T18:24:40.771747+0000","last_active":"2023-12-14T18:25:37.839687+0000","last_peered":"2023-12-14T18:25:37.839687+0000","last_clean":"2023-12-14T18:25:37.839687+0000","last_became_active":"2023-12-14T18:24:40.771310+0000","last_became_peered":"2023-12-14T18:24:40.771310+0000","last_unstale":"2023-12-14T18:25:37.839687+0000","last_undegraded":"2023-12-14T18:25:37.839687+0000","last_fullsized":"2023-12-14T18:25:37.839687+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:16:09.218784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"84'5","reported_seq":43,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.853179+0000","last_change":"2023-12-14T18:25:37.853179+0000","last_active":"2023-12-14T18:25:37.853179+0000","last_peered":"2023-12-14T18:25:37.853179+0000","last_clean":"2023-12-14T18:25:37.853179+0000","last_became_active":"2023-12-14T18:25:37.848064+0000","last_became_peered":"2023-12-14T18:25:37.848064+0000","last_unstale":"2023-12-14T18:25:37.853179+0000","last_undegraded":"2023-12-14T18:25:37.853179+0000","last_fullsized":"2023-12-14T18:25:37.853179+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:47:58.918839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":7,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,3],"acting":[9,11,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840322+0000","last_change":"2023-12-14T18:24:36.594938+0000","last_active":"2023-12-14T18:25:37.840322+0000","last_peered":"2023-12-14T18:25:37.840322+0000","last_clean":"2023-12-14T18:25:37.840322+0000","last_became_active":"2023-12-14T18:24:36.594573+0000","last_became_peered":"2023-12-14T18:24:36.594573+0000","last_unstale":"2023-12-14T18:25:37.840322+0000","last_undegraded":"2023-12-14T18:25:37.840322+0000","last_fullsized":"2023-12-14T18:25:37.840322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:32:38.255137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840467+0000","last_change":"2023-12-14T18:24:40.601146+0000","last_active":"2023-12-14T18:25:37.840467+0000","last_peered":"2023-12-14T18:25:37.840467+0000","last_clean":"2023-12-14T18:25:37.840467+0000","last_became_active":"2023-12-14T18:24:40.600745+0000","last_became_peered":"2023-12-14T18:24:40.600745+0000","last_unstale":"2023-12-14T18:25:37.840467+0000","last_undegraded":"2023-12-14T18:25:37.840467+0000","last_fullsized":"2023-12-14T18:25:37.840467+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:19:54.844673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'4","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844456+0000","last_change":"2023-12-14T18:24:36.175880+0000","last_active":"2023-12-14T18:25:37.844456+0000","last_peered":"2023-12-14T18:25:37.844456+0000","last_clean":"2023-12-14T18:25:37.844456+0000","last_became_active":"2023-12-14T18:24:36.175514+0000","last_became_peered":"2023-12-14T18:24:36.175514+0000","last_unstale":"2023-12-14T18:25:37.844456+0000","last_undegraded":"2023-12-14T18:25:37.844456+0000","last_fullsized":"2023-12-14T18:25:37.844456+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:31:53.320299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052652+0000","last_change":"2023-12-14T18:24:36.577062+0000","last_active":"2023-12-14T18:25:38.052652+0000","last_peered":"2023-12-14T18:25:38.052652+0000","last_clean":"2023-12-14T18:25:38.052652+0000","last_became_active":"2023-12-14T18:24:36.576733+0000","last_became_peered":"2023-12-14T18:24:36.576733+0000","last_unstale":"2023-12-14T18:25:38.052652+0000","last_undegraded":"2023-12-14T18:25:38.052652+0000","last_fullsized":"2023-12-14T18:25:38.052652+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:09:58.268670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844112+0000","last_change":"2023-12-14T18:24:40.772752+0000","last_active":"2023-12-14T18:25:37.844112+0000","last_peered":"2023-12-14T18:25:37.844112+0000","last_clean":"2023-12-14T18:25:37.844112+0000","last_became_active":"2023-12-14T18:24:40.772168+0000","last_became_peered":"2023-12-14T18:24:40.772168+0000","last_unstale":"2023-12-14T18:25:37.844112+0000","last_undegraded":"2023-12-14T18:25:37.844112+0000","last_fullsized":"2023-12-14T18:25:37.844112+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:12:47.177023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"84'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840372+0000","last_change":"2023-12-14T18:24:36.178885+0000","last_active":"2023-12-14T18:25:37.840372+0000","last_peered":"2023-12-14T18:25:37.840372+0000","last_clean":"2023-12-14T18:25:37.840372+0000","last_became_active":"2023-12-14T18:24:36.178386+0000","last_became_peered":"2023-12-14T18:24:36.178386+0000","last_unstale":"2023-12-14T18:25:37.840372+0000","last_undegraded":"2023-12-14T18:25:37.840372+0000","last_fullsized":"2023-12-14T18:25:37.840372+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:19:35.071421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842574+0000","last_change":"2023-12-14T18:24:36.590037+0000","last_active":"2023-12-14T18:25:37.842574+0000","last_peered":"2023-12-14T18:25:37.842574+0000","last_clean":"2023-12-14T18:25:37.842574+0000","last_became_active":"2023-12-14T18:24:36.589659+0000","last_became_peered":"2023-12-14T18:24:36.589659+0000","last_unstale":"2023-12-14T18:25:37.842574+0000","last_undegraded":"2023-12-14T18:25:37.842574+0000","last_fullsized":"2023-12-14T18:25:37.842574+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:09:42.198821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":12,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.851489+0000","last_change":"2023-12-14T18:25:37.850532+0000","last_active":"2023-12-14T18:25:37.851489+0000","last_peered":"2023-12-14T18:25:37.851489+0000","last_clean":"2023-12-14T18:25:37.851489+0000","last_became_active":"2023-12-14T18:25:37.850082+0000","last_became_peered":"2023-12-14T18:25:37.850082+0000","last_unstale":"2023-12-14T18:25:37.851489+0000","last_undegraded":"2023-12-14T18:25:37.851489+0000","last_fullsized":"2023-12-14T18:25:37.851489+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T19:24:47.887065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,2,5],"acting":[8,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"84'3","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839807+0000","last_change":"2023-12-14T18:24:36.173350+0000","last_active":"2023-12-14T18:25:37.839807+0000","last_peered":"2023-12-14T18:25:37.839807+0000","last_clean":"2023-12-14T18:25:37.839807+0000","last_became_active":"2023-12-14T18:24:36.172840+0000","last_became_peered":"2023-12-14T18:24:36.172840+0000","last_unstale":"2023-12-14T18:25:37.839807+0000","last_undegraded":"2023-12-14T18:25:37.839807+0000","last_fullsized":"2023-12-14T18:25:37.839807+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:03:03.230376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842563+0000","last_change":"2023-12-14T18:24:36.592104+0000","last_active":"2023-12-14T18:25:37.842563+0000","last_peered":"2023-12-14T18:25:37.842563+0000","last_clean":"2023-12-14T18:25:37.842563+0000","last_became_active":"2023-12-14T18:24:36.591568+0000","last_became_peered":"2023-12-14T18:24:36.591568+0000","last_unstale":"2023-12-14T18:25:37.842563+0000","last_undegraded":"2023-12-14T18:25:37.842563+0000","last_fullsized":"2023-12-14T18:25:37.842563+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:38:19.772980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"84'1","reported_seq":29,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.059912+0000","last_change":"2023-12-14T18:25:38.059912+0000","last_active":"2023-12-14T18:25:38.059912+0000","last_peered":"2023-12-14T18:25:38.059912+0000","last_clean":"2023-12-14T18:25:38.059912+0000","last_became_active":"2023-12-14T18:25:38.054942+0000","last_became_peered":"2023-12-14T18:25:38.054942+0000","last_unstale":"2023-12-14T18:25:38.059912+0000","last_undegraded":"2023-12-14T18:25:38.059912+0000","last_fullsized":"2023-12-14T18:25:38.059912+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T18:35:48.203934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":123,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,7,8],"acting":[4,10,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"84'1","reported_seq":15,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.856268+0000","last_change":"2023-12-14T18:25:37.855817+0000","last_active":"2023-12-14T18:25:37.856268+0000","last_peered":"2023-12-14T18:25:37.856268+0000","last_clean":"2023-12-14T18:25:37.856268+0000","last_became_active":"2023-12-14T18:25:37.849192+0000","last_became_peered":"2023-12-14T18:25:37.849192+0000","last_unstale":"2023-12-14T18:25:37.856268+0000","last_undegraded":"2023-12-14T18:25:37.856268+0000","last_fullsized":"2023-12-14T18:25:37.856268+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:23:11.825835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,3,8],"acting":[11,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840095+0000","last_change":"2023-12-14T18:24:36.587777+0000","last_active":"2023-12-14T18:25:37.840095+0000","last_peered":"2023-12-14T18:25:37.840095+0000","last_clean":"2023-12-14T18:25:37.840095+0000","last_became_active":"2023-12-14T18:24:36.587317+0000","last_became_peered":"2023-12-14T18:24:36.587317+0000","last_unstale":"2023-12-14T18:25:37.840095+0000","last_undegraded":"2023-12-14T18:25:37.840095+0000","last_fullsized":"2023-12-14T18:25:37.840095+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:22:21.777360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844980+0000","last_change":"2023-12-14T18:24:40.771913+0000","last_active":"2023-12-14T18:25:37.844980+0000","last_peered":"2023-12-14T18:25:37.844980+0000","last_clean":"2023-12-14T18:25:37.844980+0000","last_became_active":"2023-12-14T18:24:40.771425+0000","last_became_peered":"2023-12-14T18:24:40.771425+0000","last_unstale":"2023-12-14T18:25:37.844980+0000","last_undegraded":"2023-12-14T18:25:37.844980+0000","last_fullsized":"2023-12-14T18:25:37.844980+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:20:55.992683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'4","reported_seq":31,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842694+0000","last_change":"2023-12-14T18:24:36.176801+0000","last_active":"2023-12-14T18:25:37.842694+0000","last_peered":"2023-12-14T18:25:37.842694+0000","last_clean":"2023-12-14T18:25:37.842694+0000","last_became_active":"2023-12-14T18:24:36.176001+0000","last_became_peered":"2023-12-14T18:24:36.176001+0000","last_unstale":"2023-12-14T18:25:37.842694+0000","last_undegraded":"2023-12-14T18:25:37.842694+0000","last_fullsized":"2023-12-14T18:25:37.842694+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:31:08.426978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845322+0000","last_change":"2023-12-14T18:24:36.592898+0000","last_active":"2023-12-14T18:25:37.845322+0000","last_peered":"2023-12-14T18:25:37.845322+0000","last_clean":"2023-12-14T18:25:37.845322+0000","last_became_active":"2023-12-14T18:24:36.586650+0000","last_became_peered":"2023-12-14T18:24:36.586650+0000","last_unstale":"2023-12-14T18:25:37.845322+0000","last_undegraded":"2023-12-14T18:25:37.845322+0000","last_fullsized":"2023-12-14T18:25:37.845322+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:35:59.247019+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840411+0000","last_change":"2023-12-14T18:24:40.599181+0000","last_active":"2023-12-14T18:25:37.840411+0000","last_peered":"2023-12-14T18:25:37.840411+0000","last_clean":"2023-12-14T18:25:37.840411+0000","last_became_active":"2023-12-14T18:24:40.598785+0000","last_became_peered":"2023-12-14T18:24:40.598785+0000","last_unstale":"2023-12-14T18:25:37.840411+0000","last_undegraded":"2023-12-14T18:25:37.840411+0000","last_fullsized":"2023-12-14T18:25:37.840411+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T05:55:55.159561+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"84'3","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841700+0000","last_change":"2023-12-14T18:24:36.173644+0000","last_active":"2023-12-14T18:25:37.841700+0000","last_peered":"2023-12-14T18:25:37.841700+0000","last_clean":"2023-12-14T18:25:37.841700+0000","last_became_active":"2023-12-14T18:24:36.173059+0000","last_became_peered":"2023-12-14T18:24:36.173059+0000","last_unstale":"2023-12-14T18:25:37.841700+0000","last_undegraded":"2023-12-14T18:25:37.841700+0000","last_fullsized":"2023-12-14T18:25:37.841700+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:16:27.909550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844525+0000","last_change":"2023-12-14T18:24:36.588761+0000","last_active":"2023-12-14T18:25:37.844525+0000","last_peered":"2023-12-14T18:25:37.844525+0000","last_clean":"2023-12-14T18:25:37.844525+0000","last_became_active":"2023-12-14T18:24:36.586021+0000","last_became_peered":"2023-12-14T18:24:36.586021+0000","last_unstale":"2023-12-14T18:25:37.844525+0000","last_undegraded":"2023-12-14T18:25:37.844525+0000","last_fullsized":"2023-12-14T18:25:37.844525+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:45:41.729490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052349+0000","last_change":"2023-12-14T18:24:40.607286+0000","last_active":"2023-12-14T18:25:38.052349+0000","last_peered":"2023-12-14T18:25:38.052349+0000","last_clean":"2023-12-14T18:25:38.052349+0000","last_became_active":"2023-12-14T18:24:40.606925+0000","last_became_peered":"2023-12-14T18:24:40.606925+0000","last_unstale":"2023-12-14T18:25:38.052349+0000","last_undegraded":"2023-12-14T18:25:38.052349+0000","last_fullsized":"2023-12-14T18:25:38.052349+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T23:26:27.152905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"79'1","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.847697+0000","last_change":"2023-12-14T18:24:36.174467+0000","last_active":"2023-12-14T18:25:37.847697+0000","last_peered":"2023-12-14T18:25:37.847697+0000","last_clean":"2023-12-14T18:25:37.847697+0000","last_became_active":"2023-12-14T18:24:36.173920+0000","last_became_peered":"2023-12-14T18:24:36.173920+0000","last_unstale":"2023-12-14T18:25:37.847697+0000","last_undegraded":"2023-12-14T18:25:37.847697+0000","last_fullsized":"2023-12-14T18:25:37.847697+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:23:20.860799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.846258+0000","last_change":"2023-12-14T18:24:36.597537+0000","last_active":"2023-12-14T18:25:37.846258+0000","last_peered":"2023-12-14T18:25:37.846258+0000","last_clean":"2023-12-14T18:25:37.846258+0000","last_became_active":"2023-12-14T18:24:36.594599+0000","last_became_peered":"2023-12-14T18:24:36.594599+0000","last_unstale":"2023-12-14T18:25:37.846258+0000","last_undegraded":"2023-12-14T18:25:37.846258+0000","last_fullsized":"2023-12-14T18:25:37.846258+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T06:00:18.245087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":17,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844725+0000","last_change":"2023-12-14T18:24:40.597990+0000","last_active":"2023-12-14T18:25:37.844725+0000","last_peered":"2023-12-14T18:25:37.844725+0000","last_clean":"2023-12-14T18:25:37.844725+0000","last_became_active":"2023-12-14T18:24:40.597568+0000","last_became_peered":"2023-12-14T18:24:40.597568+0000","last_unstale":"2023-12-14T18:25:37.844725+0000","last_undegraded":"2023-12-14T18:25:37.844725+0000","last_fullsized":"2023-12-14T18:25:37.844725+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:39.569238+0000","last_clean_scrub_stamp":"2023-12-14T18:24:39.569238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:36:17.954844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"79'2","reported_seq":28,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844241+0000","last_change":"2023-12-14T18:24:36.173141+0000","last_active":"2023-12-14T18:25:37.844241+0000","last_peered":"2023-12-14T18:25:37.844241+0000","last_clean":"2023-12-14T18:25:37.844241+0000","last_became_active":"2023-12-14T18:24:36.172811+0000","last_became_peered":"2023-12-14T18:24:36.172811+0000","last_unstale":"2023-12-14T18:25:37.844241+0000","last_undegraded":"2023-12-14T18:25:37.844241+0000","last_fullsized":"2023-12-14T18:25:37.844241+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:35.142678+0000","last_clean_scrub_stamp":"2023-12-14T18:24:35.142678+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T22:39:02.206044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.844789+0000","last_change":"2023-12-14T18:24:36.587333+0000","last_active":"2023-12-14T18:25:37.844789+0000","last_peered":"2023-12-14T18:25:37.844789+0000","last_clean":"2023-12-14T18:25:37.844789+0000","last_became_active":"2023-12-14T18:24:36.586914+0000","last_became_peered":"2023-12-14T18:24:36.586914+0000","last_unstale":"2023-12-14T18:25:37.844789+0000","last_undegraded":"2023-12-14T18:25:37.844789+0000","last_fullsized":"2023-12-14T18:25:37.844789+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:48:31.035981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.842775+0000","last_change":"2023-12-14T18:24:36.591896+0000","last_active":"2023-12-14T18:25:37.842775+0000","last_peered":"2023-12-14T18:25:37.842775+0000","last_clean":"2023-12-14T18:25:37.842775+0000","last_became_active":"2023-12-14T18:24:36.591097+0000","last_became_peered":"2023-12-14T18:24:36.591097+0000","last_unstale":"2023-12-14T18:25:37.842775+0000","last_undegraded":"2023-12-14T18:25:37.842775+0000","last_fullsized":"2023-12-14T18:25:37.842775+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T04:22:08.844018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841677+0000","last_change":"2023-12-14T18:24:36.581688+0000","last_active":"2023-12-14T18:25:37.841677+0000","last_peered":"2023-12-14T18:25:37.841677+0000","last_clean":"2023-12-14T18:25:37.841677+0000","last_became_active":"2023-12-14T18:24:36.580660+0000","last_became_peered":"2023-12-14T18:24:36.580660+0000","last_unstale":"2023-12-14T18:25:37.841677+0000","last_undegraded":"2023-12-14T18:25:37.841677+0000","last_fullsized":"2023-12-14T18:25:37.841677+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:11:30.240797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840500+0000","last_change":"2023-12-14T18:24:36.583025+0000","last_active":"2023-12-14T18:25:37.840500+0000","last_peered":"2023-12-14T18:25:37.840500+0000","last_clean":"2023-12-14T18:25:37.840500+0000","last_became_active":"2023-12-14T18:24:36.582510+0000","last_became_peered":"2023-12-14T18:24:36.582510+0000","last_unstale":"2023-12-14T18:25:37.840500+0000","last_undegraded":"2023-12-14T18:25:37.840500+0000","last_fullsized":"2023-12-14T18:25:37.840500+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T21:34:09.434901+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.052493+0000","last_change":"2023-12-14T18:24:36.579383+0000","last_active":"2023-12-14T18:25:38.052493+0000","last_peered":"2023-12-14T18:25:38.052493+0000","last_clean":"2023-12-14T18:25:38.052493+0000","last_became_active":"2023-12-14T18:24:36.579037+0000","last_became_peered":"2023-12-14T18:24:36.579037+0000","last_unstale":"2023-12-14T18:25:38.052493+0000","last_undegraded":"2023-12-14T18:25:38.052493+0000","last_fullsized":"2023-12-14T18:25:38.052493+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:20:41.833469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":30,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:38.053594+0000","last_change":"2023-12-14T18:25:38.053594+0000","last_active":"2023-12-14T18:25:38.053594+0000","last_peered":"2023-12-14T18:25:38.053594+0000","last_clean":"2023-12-14T18:25:38.053594+0000","last_became_active":"2023-12-14T18:25:38.053317+0000","last_became_peered":"2023-12-14T18:25:38.053317+0000","last_unstale":"2023-12-14T18:25:38.053594+0000","last_undegraded":"2023-12-14T18:25:38.053594+0000","last_fullsized":"2023-12-14T18:25:38.053594+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T03:19:29.086598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843497+0000","last_change":"2023-12-14T18:24:36.578613+0000","last_active":"2023-12-14T18:25:37.843497+0000","last_peered":"2023-12-14T18:25:37.843497+0000","last_clean":"2023-12-14T18:25:37.843497+0000","last_became_active":"2023-12-14T18:24:36.578287+0000","last_became_peered":"2023-12-14T18:24:36.578287+0000","last_unstale":"2023-12-14T18:25:37.843497+0000","last_undegraded":"2023-12-14T18:25:37.843497+0000","last_fullsized":"2023-12-14T18:25:37.843497+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:09:07.746792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.841403+0000","last_change":"2023-12-14T18:24:36.575078+0000","last_active":"2023-12-14T18:25:37.841403+0000","last_peered":"2023-12-14T18:25:37.841403+0000","last_clean":"2023-12-14T18:25:37.841403+0000","last_became_active":"2023-12-14T18:24:36.574658+0000","last_became_peered":"2023-12-14T18:24:36.574658+0000","last_unstale":"2023-12-14T18:25:37.841403+0000","last_undegraded":"2023-12-14T18:25:37.841403+0000","last_fullsized":"2023-12-14T18:25:37.841403+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:19:17.093417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840428+0000","last_change":"2023-12-14T18:24:36.572545+0000","last_active":"2023-12-14T18:25:37.840428+0000","last_peered":"2023-12-14T18:25:37.840428+0000","last_clean":"2023-12-14T18:25:37.840428+0000","last_became_active":"2023-12-14T18:24:36.572235+0000","last_became_peered":"2023-12-14T18:24:36.572235+0000","last_unstale":"2023-12-14T18:25:37.840428+0000","last_undegraded":"2023-12-14T18:25:37.840428+0000","last_fullsized":"2023-12-14T18:25:37.840428+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:38:19.260777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.845273+0000","last_change":"2023-12-14T18:24:36.588278+0000","last_active":"2023-12-14T18:25:37.845273+0000","last_peered":"2023-12-14T18:25:37.845273+0000","last_clean":"2023-12-14T18:25:37.845273+0000","last_became_active":"2023-12-14T18:24:36.587605+0000","last_became_peered":"2023-12-14T18:24:36.587605+0000","last_unstale":"2023-12-14T18:25:37.845273+0000","last_undegraded":"2023-12-14T18:25:37.845273+0000","last_fullsized":"2023-12-14T18:25:37.845273+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-15T20:20:58.777764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.839826+0000","last_change":"2023-12-14T18:24:36.575325+0000","last_active":"2023-12-14T18:25:37.839826+0000","last_peered":"2023-12-14T18:25:37.839826+0000","last_clean":"2023-12-14T18:25:37.839826+0000","last_became_active":"2023-12-14T18:24:36.575016+0000","last_became_peered":"2023-12-14T18:24:36.575016+0000","last_unstale":"2023-12-14T18:25:37.839826+0000","last_undegraded":"2023-12-14T18:25:37.839826+0000","last_fullsized":"2023-12-14T18:25:37.839826+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T00:41:53.442267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.840657+0000","last_change":"2023-12-14T18:24:36.585348+0000","last_active":"2023-12-14T18:25:37.840657+0000","last_peered":"2023-12-14T18:25:37.840657+0000","last_clean":"2023-12-14T18:25:37.840657+0000","last_became_active":"2023-12-14T18:24:36.584995+0000","last_became_peered":"2023-12-14T18:24:36.584995+0000","last_unstale":"2023-12-14T18:25:37.840657+0000","last_undegraded":"2023-12-14T18:25:37.840657+0000","last_fullsized":"2023-12-14T18:25:37.840657+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:11:50.337217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843900+0000","last_change":"2023-12-14T18:24:36.583571+0000","last_active":"2023-12-14T18:25:37.843900+0000","last_peered":"2023-12-14T18:25:37.843900+0000","last_clean":"2023-12-14T18:25:37.843900+0000","last_became_active":"2023-12-14T18:24:36.583266+0000","last_became_peered":"2023-12-14T18:24:36.583266+0000","last_unstale":"2023-12-14T18:25:37.843900+0000","last_undegraded":"2023-12-14T18:25:37.843900+0000","last_fullsized":"2023-12-14T18:25:37.843900+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T01:59:28.832769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":24,"reported_epoch":86,"state":"active+clean","last_fresh":"2023-12-14T18:25:37.843165+0000","last_change":"2023-12-14T18:24:36.579754+0000","last_active":"2023-12-14T18:25:37.843165+0000","last_peered":"2023-12-14T18:25:37.843165+0000","last_clean":"2023-12-14T18:25:37.843165+0000","last_became_active":"2023-12-14T18:24:36.579337+0000","last_became_peered":"2023-12-14T18:24:36.579337+0000","last_unstale":"2023-12-14T18:25:37.843165+0000","last_undegraded":"2023-12-14T18:25:37.843165+0000","last_fullsized":"2023-12-14T18:25:37.843165+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":78,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-14T18:24:36.150498+0000","last_clean_scrub_stamp":"2023-12-14T18:24:36.150498+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-16T02:17:05.199500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":246,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":123,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":311296,"data_stored":3542208,"data_compressed":212533,"data_compressed_allocated":270336,"data_compressed_original":3522560,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":64740,"num_objects":60,"num_object_clones":0,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":60,"num_whiteouts":0,"num_read":296,"num_read_kb":219,"num_write":153,"num_write_kb":109,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":5,"num_bytes_recovered":964,"num_keys_recovered":7,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":194220,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":135,"ondisk_log_size":135,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":75,"seq":322122547224,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6112,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":5184,"kb_avail":93739040,"statfs":{"total":95995035648,"available":95988776960,"internally_reserved":0,"allocated":942080,"data_stored":477716,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5308416},"hb_peers":[0,1,2,3,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.66100000000000003}]},{"osd":1,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.61399999999999999,"5min":0.61399999999999999,"15min":0.61399999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.88,"5min":0.88,"15min":0.88},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.78400000000000003}]},{"osd":2,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.70499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":4,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.94699999999999995,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.68999999999999995,"5min":0.68999999999999995,"15min":0.68999999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.74299999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":1.024,"5min":1.024,"15min":1.024},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.56799999999999995}]},{"osd":6,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"max":{"1min":1.3480000000000001,"5min":1.3480000000000001,"15min":1.3480000000000001},"last":0.77200000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"max":{"1min":1.147,"5min":1.147,"15min":1.147},"last":0.82099999999999995}]},{"osd":8,"last update":"Thu Dec 14 18:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.71099999999999997,"15min":0.71099999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"last":0.79900000000000004}]},{"osd":9,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"max":{"1min":1.4790000000000001,"5min":1.4790000000000001,"15min":1.4790000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.83499999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.86899999999999999}]}]},{"osd":10,"up_from":67,"seq":287762808860,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6572,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738580,"statfs":{"total":95995035648,"available":95988305920,"internally_reserved":0,"allocated":1085440,"data_stored":1129537,"data_compressed":53238,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":6.109,"5min":6.109,"15min":6.109},"last":0.35699999999999998},{"interface":"front","average":{"1min":1.0249999999999999,"5min":1.0249999999999999,"15min":1.0249999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":9.5180000000000007,"5min":9.5180000000000007,"15min":9.5180000000000007},"last":0.38}]},{"osd":1,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":8.6690000000000005,"5min":8.6690000000000005,"15min":8.6690000000000005},"last":0.499},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.33500000000000002}]},{"osd":2,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73899999999999999,"15min":0.73899999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":3.0249999999999999,"5min":3.0249999999999999,"15min":3.0249999999999999},"last":0.54100000000000004},{"interface":"front","average":{"1min":1.095,"5min":1.095,"15min":1.095},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":9.5459999999999994,"5min":9.5459999999999994,"15min":9.5459999999999994},"last":0.65300000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":9.4380000000000006,"5min":9.4380000000000006,"15min":9.4380000000000006},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":1.802,"5min":1.802,"15min":1.802},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":5.9470000000000001,"5min":5.9470000000000001,"15min":5.9470000000000001},"last":0.441},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":6.1470000000000002,"5min":6.1470000000000002,"15min":6.1470000000000002},"last":0.45300000000000001}]},{"osd":5,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.76900000000000002,"15min":0.76900000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.359,"5min":2.359,"15min":2.359},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.46600000000000003,"5min":0.46600000000000003,"15min":0.46600000000000003},"max":{"1min":2.7890000000000001,"5min":2.7890000000000001,"15min":2.7890000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":9.4830000000000005,"5min":9.4830000000000005,"15min":9.4830000000000005},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":2.1440000000000001,"5min":2.1440000000000001,"15min":2.1440000000000001},"last":0.52800000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44600000000000001}]},{"osd":8,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81699999999999995},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":2.202,"5min":2.202,"15min":2.202},"last":0.624},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.4590000000000001,"5min":1.4590000000000001,"15min":1.4590000000000001},"last":0.63700000000000001}]},{"osd":9,"last update":"Thu Dec 14 18:24:45 2023","interfaces":[{"interface":"back","average":{"1min":0.82699999999999996,"5min":0.82699999999999996,"15min":0.82699999999999996},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":2.1749999999999998,"5min":2.1749999999999998,"15min":2.1749999999999998},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"min":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"max":{"1min":2.403,"5min":2.403,"15min":2.403},"last":0.67300000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:25:03 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.629,"5min":0.629,"15min":0.629},"max":{"1min":2.6400000000000001,"5min":2.6400000000000001,"15min":2.6400000000000001},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":0.98799999999999999},"min":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"max":{"1min":6.0309999999999997,"5min":6.0309999999999997,"15min":6.0309999999999997},"last":0.71999999999999997}]}]},{"osd":9,"up_from":62,"seq":266287972382,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6320,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5376,"kb_avail":93738832,"statfs":{"total":95995035648,"available":95988563968,"internally_reserved":0,"allocated":958464,"data_stored":477952,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5505024},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.42199999999999999,"15min":0.42999999999999999},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.53500000000000003,"5min":0.76900000000000002,"15min":0.76900000000000002},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.49099999999999999,"15min":0.505},"min":{"1min":0.27200000000000002,"5min":0.254,"15min":0.254},"max":{"1min":0.73999999999999999,"5min":0.94699999999999995,"15min":0.94699999999999995},"last":0.36099999999999999}]},{"osd":1,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.60999999999999999,"15min":0.62},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.83399999999999996,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.63900000000000001,"15min":0.64700000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.495}]},{"osd":2,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.58899999999999997,"15min":0.59399999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.76300000000000001,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.61199999999999999,"15min":0.61899999999999999},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.91200000000000003,"5min":1.0269999999999999,"15min":1.0269999999999999},"last":0.45500000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.46100000000000002,"15min":0.47199999999999998},"min":{"1min":0.27700000000000002,"5min":0.27700000000000002,"15min":0.27700000000000002},"max":{"1min":0.69699999999999995,"5min":0.79900000000000004,"15min":0.79900000000000004},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.48799999999999999,"15min":0.505},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.59199999999999997,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.28399999999999997}]},{"osd":4,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.65600000000000003,"15min":0.67200000000000004},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.879,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.63200000000000001,"15min":0.63700000000000001},"min":{"1min":0.45500000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.76000000000000001,"5min":0.83299999999999996,"15min":0.83299999999999996},"last":0.42699999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.58899999999999997,"15min":0.58799999999999997},"min":{"1min":0.44,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.72999999999999998,"5min":0.96599999999999997,"15min":0.96599999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.63800000000000001,"15min":0.64000000000000001},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.89900000000000002,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.52000000000000002}]},{"osd":6,"last update":"Thu Dec 14 18:24:27 2023","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.012,"5min":1.012,"15min":1.012},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.67400000000000004,"15min":0.67400000000000004},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.0860000000000001,"5min":1.0860000000000001,"15min":1.0860000000000001},"last":0.41499999999999998}]},{"osd":7,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.60599999999999998,"5min":0.66900000000000004,"15min":0.67900000000000005},"min":{"1min":0.45900000000000002,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.80700000000000005,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.73699999999999999,"15min":0.749},"min":{"1min":0.502,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.84799999999999998,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.441}]},{"osd":8,"last update":"Thu Dec 14 18:25:28 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.72199999999999998,"15min":0.73799999999999999},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.81699999999999995,"5min":1.214,"15min":1.214},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.626,"5min":0.67500000000000004,"15min":0.68300000000000005},"min":{"1min":0.47199999999999998,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.79600000000000004,"5min":1.071,"15min":1.071},"last":0.59599999999999997}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.70499999999999996,"15min":0.70499999999999996},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.0169999999999999},"last":0.54900000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:25:42 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.66200000000000003,"15min":0.66200000000000003},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.55900000000000005}]}]},{"osd":8,"up_from":55,"seq":236223201315,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6444,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":5504,"kb_avail":93738708,"statfs":{"total":95995035648,"available":95988436992,"internally_reserved":0,"allocated":954368,"data_stored":477541,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5636096},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.435,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.77900000000000003,"5min":0.79800000000000004,"15min":0.79800000000000004},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.52200000000000002,"15min":0.52000000000000002},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":0.81899999999999995,"5min":0.81899999999999995,"15min":0.81899999999999995},"last":0.45400000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.60199999999999998,"15min":0.59999999999999998},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57699999999999996,"15min":0.57199999999999995},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.44800000000000001,"15min":0.44400000000000001},"min":{"1min":0.32100000000000001,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45000000000000001,"15min":0.44900000000000001},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"last":0.42699999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.56399999999999995,"15min":0.56699999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.76500000000000001,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.57499999999999996,"15min":0.57099999999999995},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.85199999999999998,"5min":0.85199999999999998,"15min":0.85199999999999998},"last":0.40699999999999997}]},{"osd":4,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.629,"15min":0.627},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.63200000000000001,"15min":0.628},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.73699999999999999}]},{"osd":5,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.379,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.54700000000000004,"15min":0.54100000000000004},"min":{"1min":0.35799999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.60499999999999998}]},{"osd":6,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.65400000000000003,"15min":0.65200000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.626},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.67100000000000004,"15min":0.66800000000000004},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.042},"last":0.65000000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:25:11 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.68500000000000005,"15min":0.68100000000000005},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.69999999999999996,"15min":0.69399999999999995},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":0.90100000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.80200000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.68799999999999994,"15min":0.68200000000000005},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.754,"5min":0.69499999999999995,"15min":0.68500000000000005},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.81699999999999995}]},{"osd":10,"last update":"Thu Dec 14 18:24:50 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.751},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":0.89000000000000001,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.84399999999999997}]},{"osd":11,"last update":"Thu Dec 14 18:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":0.97999999999999998},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"min":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"max":{"1min":1.018,"5min":1.018,"15min":1.018},"last":0.77900000000000003}]}]},{"osd":7,"up_from":48,"seq":206158430246,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6556,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738596,"statfs":{"total":95995035648,"available":95988322304,"internally_reserved":0,"allocated":1003520,"data_stored":1073353,"data_compressed":53238,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:23:53 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.434}]},{"osd":1,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.40699999999999997,"15min":0.39400000000000002},"min":{"1min":0.32400000000000001,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.74,"5min":1.74,"15min":1.74},"last":0.441},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.433,"15min":0.42299999999999999},"min":{"1min":0.33100000000000002,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":1.5940000000000001,"5min":1.5940000000000001,"15min":1.5940000000000001},"last":0.42099999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.56100000000000005,"15min":0.54800000000000004},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.629,"5min":0.54900000000000004,"15min":0.53600000000000003},"min":{"1min":0.443,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.94999999999999996,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.58299999999999996}]},{"osd":3,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.42699999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.59699999999999998,"5min":0.56699999999999995,"15min":0.56200000000000006},"min":{"1min":0.44800000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.77600000000000002,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.60999999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.53600000000000003,"15min":0.48999999999999999},"min":{"1min":0.29499999999999998,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":5.7830000000000004,"5min":5.7830000000000004,"15min":5.7830000000000004},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.49299999999999999,"15min":0.48299999999999998},"min":{"1min":0.35699999999999998,"5min":0.308,"15min":0.308},"max":{"1min":1.536,"5min":1.536,"15min":1.536},"last":0.47599999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.59399999999999997,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":1.605,"5min":1.605,"15min":1.605},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.60099999999999998,"15min":0.58999999999999997},"min":{"1min":0.53100000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.73399999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.63900000000000001,"15min":0.63600000000000001},"min":{"1min":0.52000000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.77500000000000002,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63100000000000001,"15min":0.624},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.659,"5min":1.659,"15min":1.659},"last":0.749}]},{"osd":8,"last update":"Thu Dec 14 18:25:10 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.58999999999999997,"5min":0.46000000000000002,"15min":0.46000000000000002},"max":{"1min":0.997,"5min":0.997,"15min":0.997},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.57299999999999995,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.82699999999999996,"5min":0.879,"15min":0.879},"last":0.68799999999999994}]},{"osd":9,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.747,"15min":0.74299999999999999},"min":{"1min":0.56399999999999995,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":2.4140000000000001,"5min":2.4140000000000001,"15min":2.4140000000000001},"last":0.627},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.71199999999999997},"min":{"1min":0.56399999999999995,"5min":0.498,"15min":0.498},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.79700000000000004}]},{"osd":10,"last update":"Thu Dec 14 18:25:47 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.77400000000000002,"15min":0.78400000000000003},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.92900000000000005,"5min":1.6220000000000001,"15min":1.6220000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.747,"5min":0.79900000000000004,"15min":0.80800000000000005},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.125,"5min":1.8120000000000001,"15min":1.8120000000000001},"last":0.81899999999999995}]},{"osd":11,"last update":"Thu Dec 14 18:25:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"min":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.65300000000000002,"5min":0.65300000000000002,"15min":0.65300000000000002},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.78300000000000003}]}]},{"osd":6,"up_from":43,"seq":184683593768,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6364,"kb_used_data":916,"kb_used_omap":0,"kb_used_meta":5440,"kb_avail":93738788,"statfs":{"total":95995035648,"available":95988518912,"internally_reserved":0,"allocated":937984,"data_stored":473128,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5570560},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.38700000000000001,"15min":0.38500000000000001},"min":{"1min":0.26300000000000001,"5min":0.215,"15min":0.215},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.255},{"interface":"front","average":{"1min":0.39800000000000002,"5min":0.39700000000000002,"15min":0.39500000000000002},"min":{"1min":0.28299999999999997,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53600000000000003,"15min":0.53000000000000003},"min":{"1min":0.34000000000000002,"5min":0.253,"15min":0.253},"max":{"1min":0.97699999999999998,"5min":0.97699999999999998,"15min":0.97699999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53300000000000003,"15min":0.53100000000000003},"min":{"1min":0.39700000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45300000000000001}]},{"osd":2,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.56599999999999995,"15min":0.56399999999999995},"min":{"1min":0.45900000000000002,"5min":0.308,"15min":0.308},"max":{"1min":1.2849999999999999,"5min":1.2849999999999999,"15min":1.2849999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.55700000000000005,"15min":0.55300000000000005},"min":{"1min":0.41599999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.88800000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45700000000000002,"15min":0.46100000000000002},"min":{"1min":0.314,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.81599999999999995,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.37},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.47799999999999998,"15min":0.47599999999999998},"min":{"1min":0.36599999999999999,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":1.111,"5min":1.111,"15min":1.111},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.58199999999999996,"15min":0.57599999999999996},"min":{"1min":0.36299999999999999,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58399999999999996,"15min":0.58699999999999997},"min":{"1min":0.44600000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.86099999999999999,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.41999999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.61899999999999999,"15min":0.627},"min":{"1min":0.41999999999999998,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.62,"15min":0.63100000000000001},"min":{"1min":0.39400000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.68799999999999994}]},{"osd":7,"last update":"Thu Dec 14 18:24:58 2023","interfaces":[{"interface":"back","average":{"1min":0.63800000000000001,"5min":0.622,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80900000000000005,"5min":0.81599999999999995,"15min":0.81599999999999995},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.621,"5min":0.624,"15min":0.624},"min":{"1min":0.42799999999999999,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"last":0.49099999999999999}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.623,"5min":0.65200000000000002,"15min":0.65700000000000003},"min":{"1min":0.46700000000000003,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.79200000000000004,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.627,"5min":0.65100000000000002,"15min":0.65500000000000003},"min":{"1min":0.42799999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.79800000000000004,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.59299999999999997}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.54700000000000004,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.876,"5min":0.876,"15min":0.876},"last":0.52100000000000002},{"interface":"front","average":{"1min":0.51400000000000001,"5min":0.63600000000000001,"15min":0.65600000000000003},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.83299999999999996,"5min":0.85299999999999998,"15min":0.85299999999999998},"last":0.623}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"last":0.51200000000000001}]},{"osd":11,"last update":"Thu Dec 14 18:25:43 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.96399999999999997,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.74299999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574908,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7388,"kb_used_data":980,"kb_used_omap":0,"kb_used_meta":6400,"kb_avail":93737764,"statfs":{"total":95995035648,"available":95987470336,"internally_reserved":0,"allocated":1003520,"data_stored":533874,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6553600},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.48399999999999999,"15min":0.46200000000000002},"min":{"1min":0.36099999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.73999999999999999,"5min":0.76100000000000001,"15min":0.76100000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.49099999999999999,"15min":0.46300000000000002},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"last":0.628}]},{"osd":2,"last update":"Thu Dec 14 18:25:34 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.52700000000000002,"15min":0.51000000000000001},"min":{"1min":0.42999999999999999,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.53600000000000003,"15min":0.52100000000000002},"min":{"1min":0.41299999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.70199999999999996}]},{"osd":3,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.51700000000000002,"5min":0.47799999999999998,"15min":0.46500000000000002},"min":{"1min":0.29399999999999998,"5min":0.29399999999999998,"15min":0.29399999999999998},"max":{"1min":0.80200000000000005,"5min":0.80200000000000005,"15min":0.80200000000000005},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.505,"15min":0.48699999999999999},"min":{"1min":0.34000000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.52700000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.42799999999999999,"15min":0.40799999999999997},"min":{"1min":0.316,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.47499999999999998,"5min":0.42399999999999999,"15min":0.40300000000000002},"min":{"1min":0.33500000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77600000000000002,"5min":1.542,"15min":1.542},"last":0.46500000000000002}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.53800000000000003,"15min":0.51900000000000002},"min":{"1min":0.442,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":0.96899999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.56399999999999995,"15min":0.54900000000000004},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Dec 14 18:25:40 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.57999999999999996,"15min":0.56599999999999995},"min":{"1min":0.433,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.92000000000000004,"5min":0.92000000000000004,"15min":0.92000000000000004},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55400000000000005,"15min":0.54500000000000004},"min":{"1min":0.433,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.80300000000000005,"5min":0.80300000000000005,"15min":0.80300000000000005},"last":0.504}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.51600000000000001,"15min":0.50600000000000001},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.52600000000000002,"15min":0.52000000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.83299999999999996,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.67700000000000005}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.61499999999999999,"15min":0.60799999999999998},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.65300000000000002,"15min":0.64200000000000002},"min":{"1min":0.501,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":0.97299999999999998,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.81100000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.68100000000000005,"15min":0.67400000000000004},"min":{"1min":0.52900000000000003,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.65800000000000003,"15min":0.64600000000000002},"min":{"1min":0.50800000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.753}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.66700000000000004,"15min":0.66700000000000004},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.89200000000000002,"5min":0.89200000000000002,"15min":0.89200000000000002},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72399999999999998,"15min":0.72399999999999998},"min":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.83399999999999996}]},{"osd":11,"last update":"Thu Dec 14 18:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.71299999999999997,"5min":0.71299999999999997,"15min":0.71299999999999997},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.84099999999999997,"15min":0.84099999999999997},"min":{"1min":0.628,"5min":0.628,"15min":0.628},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.85399999999999998}]}]},{"osd":0,"up_from":8,"seq":34359738431,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7220,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":6272,"kb_avail":93737932,"statfs":{"total":95995035648,"available":95987642368,"internally_reserved":0,"allocated":962560,"data_stored":474516,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6422528},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.49199999999999999,"15min":0.47999999999999998},"min":{"1min":0.32300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.496,"15min":0.47899999999999998},"min":{"1min":0.29999999999999999,"5min":0.222,"15min":0.222},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.56699999999999995}]},{"osd":2,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.52500000000000002,"15min":0.497},"min":{"1min":0.35899999999999999,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.54000000000000004,"15min":0.5},"min":{"1min":0.441,"5min":0.373,"15min":0.373},"max":{"1min":0.89600000000000002,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.54900000000000004}]},{"osd":3,"last update":"Thu Dec 14 18:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":7.3780000000000001,"5min":7.3780000000000001,"15min":7.3780000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.77100000000000002,"15min":0.77100000000000002},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":7.3099999999999996,"5min":7.3099999999999996,"15min":7.3099999999999996},"last":0.35999999999999999}]},{"osd":4,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.51900000000000002,"15min":0.503},"min":{"1min":0.38,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.84199999999999997,"5min":0.84199999999999997,"15min":0.84199999999999997},"last":0.629},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.51200000000000001,"15min":0.501},"min":{"1min":0.34499999999999997,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.71799999999999997,"5min":0.74199999999999999,"15min":0.74199999999999999},"last":0.72099999999999997}]},{"osd":5,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.51300000000000001,"15min":0.48599999999999999},"min":{"1min":0.32800000000000001,"5min":0.309,"15min":0.309},"max":{"1min":0.97799999999999998,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.53300000000000003,"15min":0.51100000000000001},"min":{"1min":0.40400000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.504}]},{"osd":6,"last update":"Thu Dec 14 18:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.86199999999999999,"5min":0.53900000000000003,"15min":0.49299999999999999},"min":{"1min":0.33200000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":7.1840000000000002,"5min":7.1840000000000002,"15min":7.1840000000000002},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.92600000000000005,"5min":0.56899999999999995,"15min":0.51100000000000001},"min":{"1min":0.33300000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":7.274,"5min":7.274,"15min":7.274},"last":0.59399999999999997}]},{"osd":7,"last update":"Thu Dec 14 18:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.1240000000000001,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.73199999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:25:13 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.63100000000000001,"15min":0.629},"min":{"1min":0.44700000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.67100000000000004,"5min":0.66500000000000004,"15min":0.66400000000000003},"min":{"1min":0.51200000000000001,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.64600000000000002}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.90700000000000003,"5min":0.63900000000000001,"15min":0.59499999999999997},"min":{"1min":0.33500000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":7.2290000000000001,"5min":7.2290000000000001,"15min":7.2290000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.92900000000000005,"5min":0.67300000000000004,"15min":0.63},"min":{"1min":0.40799999999999997,"5min":0.37,"15min":0.37},"max":{"1min":7.3390000000000004,"5min":7.3390000000000004,"15min":7.3390000000000004},"last":0.61899999999999999}]},{"osd":10,"last update":"Thu Dec 14 18:24:46 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.755},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.67100000000000004}]},{"osd":11,"last update":"Thu Dec 14 18:25:05 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.65500000000000003,"15min":0.65500000000000003},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.55900000000000005}]}]},{"osd":2,"up_from":18,"seq":77309411383,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":7532,"kb_used_data":996,"kb_used_omap":0,"kb_used_meta":6528,"kb_avail":93737620,"statfs":{"total":95995035648,"available":95987322880,"internally_reserved":0,"allocated":1019904,"data_stored":534602,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":6684672},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.47599999999999998,"15min":0.442},"min":{"1min":0.35599999999999998,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.48899999999999999,"15min":0.45500000000000002},"min":{"1min":0.377,"5min":0.317,"15min":0.317},"max":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"last":0.66700000000000004}]},{"osd":1,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.54600000000000004,"15min":0.48699999999999999},"min":{"1min":0.51100000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":0.98699999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.55900000000000005,"15min":0.501},"min":{"1min":0.45600000000000002,"5min":0.31,"15min":0.31},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":0.99199999999999999},"last":0.45900000000000002}]},{"osd":3,"last update":"Thu Dec 14 18:24:41 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.48199999999999998,"15min":0.46600000000000003},"min":{"1min":0.38300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.504,"15min":0.49399999999999999},"min":{"1min":0.41099999999999998,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76100000000000001,"5min":1.179,"15min":1.179},"last":0.65300000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.64900000000000002,"15min":0.64900000000000002},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"min":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Thu Dec 14 18:25:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.499,"15min":0.48099999999999998},"min":{"1min":0.35099999999999998,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.54200000000000004,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.316,"15min":0.316},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.63900000000000001,"15min":0.63},"min":{"1min":0.46100000000000002,"5min":0.442,"15min":0.442},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.61799999999999999,"15min":0.60499999999999998},"min":{"1min":0.47199999999999998,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.748}]},{"osd":7,"last update":"Thu Dec 14 18:23:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.64500000000000002,"15min":0.64500000000000002},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.53000000000000003}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.56999999999999995,"15min":0.56100000000000005},"min":{"1min":0.41099999999999998,"5min":0.38500000000000001,"15min":0.38500000000000001},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.62,"15min":0.61199999999999999},"min":{"1min":0.435,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.73099999999999998}]},{"osd":9,"last update":"Thu Dec 14 18:25:29 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.64000000000000001,"15min":0.63400000000000001},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.625},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.68799999999999994,"15min":0.67200000000000004},"min":{"1min":0.55500000000000005,"5min":0.48799999999999999,"15min":0.48799999999999999},"max":{"1min":1.2609999999999999,"5min":1.2609999999999999,"15min":1.2609999999999999},"last":0.70599999999999996}]},{"osd":10,"last update":"Thu Dec 14 18:24:48 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"min":{"1min":0.56399999999999995,"5min":0.56399999999999995,"15min":0.56399999999999995},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.78200000000000003},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52100000000000002},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.71999999999999997}]},{"osd":11,"last update":"Thu Dec 14 18:25:38 2023","interfaces":[{"interface":"back","average":{"1min":0.81499999999999995,"5min":0.81499999999999995,"15min":0.81499999999999995},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"max":{"1min":1.056,"5min":1.056,"15min":1.056},"last":0.80900000000000005}]}]},{"osd":3,"up_from":24,"seq":103079215156,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6696,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":5760,"kb_avail":93738456,"statfs":{"total":95995035648,"available":95988178944,"internally_reserved":0,"allocated":950272,"data_stored":473322,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5898240},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.33800000000000002,"15min":0.32800000000000001},"min":{"1min":0.246,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"last":0.372},{"interface":"front","average":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"min":{"1min":0.248,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"last":0.34499999999999997}]},{"osd":1,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.50800000000000001,"15min":0.48699999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.282,"5min":1.282,"15min":1.282},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.497,"15min":0.47499999999999998},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.49299999999999999}]},{"osd":2,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.53700000000000003,"15min":0.53600000000000003},"min":{"1min":0.39300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.74099999999999999,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.39300000000000002}]},{"osd":4,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.59099999999999997,"15min":0.58099999999999996},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.78400000000000003,"5min":1.214,"15min":1.214},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58299999999999996,"15min":0.57299999999999995},"min":{"1min":0.45300000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.77600000000000002,"5min":0.78200000000000003,"15min":0.78200000000000003},"last":0.57699999999999996}]},{"osd":5,"last update":"Thu Dec 14 18:25:24 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55300000000000005,"15min":0.54300000000000004},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.64100000000000001,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55600000000000005,"15min":0.54900000000000004},"min":{"1min":0.37,"5min":0.33600000000000002,"15min":0.33600000000000002},"max":{"1min":0.754,"5min":0.78900000000000003,"15min":0.78900000000000003},"last":0.54400000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.48499999999999999,"15min":0.495},"min":{"1min":0.28499999999999998,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.73699999999999999,"5min":0.755,"15min":0.755},"last":0.433},{"interface":"front","average":{"1min":0.437,"5min":0.51000000000000001,"15min":0.53000000000000003},"min":{"1min":0.34899999999999998,"5min":0.28199999999999997,"15min":0.28199999999999997},"max":{"1min":0.64500000000000002,"5min":0.77700000000000002,"15min":0.77700000000000002},"last":0.46500000000000002}]},{"osd":7,"last update":"Thu Dec 14 18:24:53 2023","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.625,"15min":0.628},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.24,"5min":1.24,"15min":1.24},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.63700000000000001,"15min":0.64300000000000002},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.81299999999999994,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.59399999999999997}]},{"osd":8,"last update":"Thu Dec 14 18:24:11 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.495},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.51900000000000002,"5min":0.51900000000000002,"15min":0.51900000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.55500000000000005}]},{"osd":9,"last update":"Thu Dec 14 18:25:31 2023","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.60299999999999998,"15min":0.61699999999999999},"min":{"1min":0.316,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79900000000000004,"5min":0.83399999999999996,"15min":0.83399999999999996},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.59699999999999998,"15min":0.61199999999999999},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.63700000000000001,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.56599999999999995}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.64900000000000002}]},{"osd":11,"last update":"Thu Dec 14 18:25:37 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65100000000000002,"15min":0.65100000000000002},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.628},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.66500000000000004}]}]},{"osd":4,"up_from":31,"seq":133143986225,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6512,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":5568,"kb_avail":93738640,"statfs":{"total":95995035648,"available":95988367360,"internally_reserved":0,"allocated":958464,"data_stored":478694,"data_compressed":17543,"data_compressed_allocated":24576,"data_compressed_original":49152,"omap_allocated":0,"internal_metadata":5701632},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:24:04 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.48799999999999999,"15min":0.48099999999999998},"min":{"1min":0.35199999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.376},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.38300000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.73299999999999998,"5min":0.73299999999999998,"15min":0.73299999999999998},"last":0.39100000000000001}]},{"osd":1,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.378,"15min":0.374},"min":{"1min":0.28699999999999998,"5min":0.25,"15min":0.25},"max":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.39700000000000002,"15min":0.38800000000000001},"min":{"1min":0.27400000000000002,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.64800000000000002,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.433}]},{"osd":2,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.54800000000000004,"15min":0.53400000000000003},"min":{"1min":0.38100000000000001,"5min":0.375,"15min":0.375},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.55600000000000005,"15min":0.54300000000000004},"min":{"1min":0.42799999999999999,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":0.81299999999999994},"last":0.61399999999999999}]},{"osd":3,"last update":"Thu Dec 14 18:25:06 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.55800000000000005,"15min":0.54800000000000004},"min":{"1min":0.48399999999999999,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.749,"5min":0.749,"15min":0.749},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.52200000000000002,"15min":0.52400000000000002},"min":{"1min":0.40699999999999997,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.40999999999999998}]},{"osd":5,"last update":"Thu Dec 14 18:25:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59499999999999997,"15min":0.57899999999999996},"min":{"1min":0.45000000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.79600000000000004,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.59299999999999997,"15min":0.58099999999999996},"min":{"1min":0.51000000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":0.76500000000000001,"5min":0.82199999999999995,"15min":0.82199999999999995},"last":0.67300000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.56000000000000005,"15min":0.55200000000000005},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.67100000000000004,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.58399999999999996,"15min":0.57199999999999995},"min":{"1min":0.437,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.72199999999999998,"5min":0.83099999999999996,"15min":0.83099999999999996},"last":0.56799999999999995}]},{"osd":7,"last update":"Thu Dec 14 18:24:55 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.53200000000000003,"15min":0.52600000000000002},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.58699999999999997,"5min":0.55900000000000005,"15min":0.55400000000000005},"min":{"1min":0.373,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.87,"5min":0.87,"15min":0.87},"last":0.47499999999999998}]},{"osd":8,"last update":"Thu Dec 14 18:25:12 2023","interfaces":[{"interface":"back","average":{"1min":0.71099999999999997,"5min":0.66700000000000004,"15min":0.65900000000000003},"min":{"1min":0.54300000000000004,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.65900000000000003,"15min":0.65100000000000002},"min":{"1min":0.503,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":0.94899999999999995,"5min":0.94899999999999995,"15min":0.94899999999999995},"last":0.70799999999999996}]},{"osd":9,"last update":"Thu Dec 14 18:25:30 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68700000000000006,"15min":0.68799999999999994},"min":{"1min":0.51900000000000002,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":0.84499999999999997,"5min":0.92200000000000004,"15min":0.92200000000000004},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.69899999999999995,"15min":0.70499999999999996},"min":{"1min":0.53200000000000003,"5min":0.375,"15min":0.375},"max":{"1min":0.85999999999999999,"5min":0.96399999999999997,"15min":0.96399999999999997},"last":0.65800000000000003}]},{"osd":10,"last update":"Thu Dec 14 18:24:49 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.71399999999999997,"15min":0.71399999999999997},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.002,"5min":1.002,"15min":1.002},"last":0.745},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.755}]},{"osd":11,"last update":"Thu Dec 14 18:25:39 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79500000000000004,"15min":0.79500000000000004},"min":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"max":{"1min":0.93200000000000005,"5min":0.93200000000000005,"15min":0.93200000000000005},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"max":{"1min":0.874,"5min":0.874,"15min":0.874},"last":0.73199999999999998}]}]},{"osd":5,"up_from":36,"seq":154618822701,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":6696,"kb_used_data":992,"kb_used_omap":0,"kb_used_meta":5696,"kb_avail":93738456,"statfs":{"total":95995035648,"available":95988178944,"internally_reserved":0,"allocated":1015808,"data_stored":1069320,"data_compressed":53238,"data_compressed_allocated":69632,"data_compressed_original":638976,"omap_allocated":0,"internal_metadata":5832704},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.41899999999999998,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69699999999999995,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.51500000000000001,"15min":0.51000000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.626,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.58399999999999996}]},{"osd":1,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.55600000000000005,"15min":0.53700000000000003},"min":{"1min":0.501,"5min":0.311,"15min":0.311},"max":{"1min":0.76700000000000002,"5min":0.89000000000000001,"15min":0.89000000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.56599999999999995,"15min":0.55000000000000004},"min":{"1min":0.45100000000000001,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.73199999999999998}]},{"osd":2,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.41699999999999998},"min":{"1min":0.314,"5min":0.254,"15min":0.254},"max":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.41999999999999998,"15min":0.40500000000000003},"min":{"1min":0.32800000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.628,"5min":0.69499999999999995,"15min":0.69499999999999995},"last":0.60699999999999998}]},{"osd":3,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.55200000000000005,"15min":0.54200000000000004},"min":{"1min":0.38200000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.72499999999999998,"5min":0.76000000000000001,"15min":0.76000000000000001},"last":0.433},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.55100000000000005,"15min":0.53600000000000003},"min":{"1min":0.41499999999999998,"5min":0.378,"15min":0.378},"max":{"1min":0.68000000000000005,"5min":0.85899999999999999,"15min":0.85899999999999999},"last":0.47799999999999998}]},{"osd":4,"last update":"Thu Dec 14 18:25:27 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.59399999999999997,"15min":0.56899999999999995},"min":{"1min":0.53900000000000003,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.78400000000000003,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.59699999999999998,"15min":0.58099999999999996},"min":{"1min":0.47599999999999998,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.81000000000000005,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.67500000000000004}]},{"osd":6,"last update":"Thu Dec 14 18:25:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60999999999999999,"15min":0.61699999999999999},"min":{"1min":0.441,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.88800000000000001,"5min":0.997,"15min":0.997},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.40600000000000003,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.90800000000000003},"last":0.54300000000000004}]},{"osd":7,"last update":"Thu Dec 14 18:24:54 2023","interfaces":[{"interface":"back","average":{"1min":0.64600000000000002,"5min":0.63600000000000001,"15min":0.63400000000000001},"min":{"1min":0.45200000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.879,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.46999999999999997,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.89500000000000002,"5min":0.96699999999999997,"15min":0.96699999999999997},"last":0.70099999999999996}]},{"osd":8,"last update":"Thu Dec 14 18:25:14 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.60499999999999998,"15min":0.59499999999999997},"min":{"1min":0.40400000000000003,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.85999999999999999,"5min":0.98499999999999999,"15min":0.98499999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.61899999999999999,"15min":0.61299999999999999},"min":{"1min":0.41299999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.82299999999999995,"5min":0.92900000000000005,"15min":0.92900000000000005},"last":0.70899999999999996}]},{"osd":9,"last update":"Thu Dec 14 18:25:33 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.58599999999999997,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.752},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.70099999999999996,"15min":0.69699999999999995},"min":{"1min":0.61799999999999999,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.72199999999999998}]},{"osd":10,"last update":"Thu Dec 14 18:25:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.72899999999999998,"15min":0.72299999999999998},"min":{"1min":0.64400000000000002,"5min":0.56299999999999994,"15min":0.56299999999999994},"max":{"1min":0.90300000000000002,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.745,"15min":0.74299999999999999},"min":{"1min":0.66900000000000004,"5min":0.58099999999999996,"15min":0.58099999999999996},"max":{"1min":0.85899999999999999,"5min":0.93700000000000006,"15min":0.93700000000000006},"last":0.74099999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34058,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35695,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":60836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":61564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":284,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2086,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":62366,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-14T18:25:50.212 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2023-12-14T18:25:50.212 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-14T18:25:50.212 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2023-12-14T18:25:50.212 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph health --format=json 2023-12-14T18:25:50.524 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:51.300 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:51 smithi093 ceph-mon[42070]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:51.300 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:51 smithi093 ceph-mon[42070]: pgmap v263: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 90 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:51.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:51 smithi097 ceph-mon[43486]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:51 smithi097 ceph-mon[43486]: pgmap v263: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 90 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:51.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.465+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3327653646 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a28104510 msgr2=0x7f7a2810c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.465+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3327653646 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a28104510 0x7f7a2810c090 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7a2000b0d0 tx=0x7f7a2000b3e0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.466+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3327653646 shutdown_connections 2023-12-14T18:25:51.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.466+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3327653646 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a28104510 0x7f7a2810c090 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.466+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3327653646 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a280ff470 0x7f7a280ff840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.466+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3327653646 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a280ffe10 0x7f7a28103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.466+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3327653646 >> 172.21.15.93:0/3327653646 conn(0x7f7a280770d0 msgr2=0x7f7a280774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:51.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.466+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3327653646 shutdown_connections 2023-12-14T18:25:51.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.466+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3327653646 wait complete. 2023-12-14T18:25:51.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.466+0000 7f7a2dadd700 1 Processor -- start 2023-12-14T18:25:51.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a2dadd700 1 -- start start 2023-12-14T18:25:51.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a2dadd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a280ff470 0x7f7a28196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a2dadd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a280ffe10 0x7f7a28196a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a2dadd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28104510 0x7f7a2819aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a2dadd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2819b4f0 con 0x7f7a280ffe10 2023-12-14T18:25:51.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a2dadd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2819b660 con 0x7f7a28104510 2023-12-14T18:25:51.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a2dadd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a2819b820 con 0x7f7a280ff470 2023-12-14T18:25:51.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a26ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a280ffe10 0x7f7a28196a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a26ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a280ffe10 0x7f7a28196a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46880/0 (socket says 172.21.15.93:46880) 2023-12-14T18:25:51.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a26ffd700 1 -- 172.21.15.93:0/3781359079 learned_addr learned my addr 172.21.15.93:0/3781359079 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:51.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a27fff700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28104510 0x7f7a2819aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a277fe700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a280ff470 0x7f7a28196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a26ffd700 1 -- 172.21.15.93:0/3781359079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a280ff470 msgr2=0x7f7a28196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a26ffd700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a280ff470 0x7f7a28196530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a26ffd700 1 -- 172.21.15.93:0/3781359079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28104510 msgr2=0x7f7a2819aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a26ffd700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28104510 0x7f7a2819aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a26ffd700 1 -- 172.21.15.93:0/3781359079 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a20009d00 con 0x7f7a280ffe10 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.467+0000 7f7a26ffd700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a280ffe10 0x7f7a28196a70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00d980 tx=0x7f7a1c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.468+0000 7f7a24ff9700 1 -- 172.21.15.93:0/3781359079 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c009b40 con 0x7f7a280ffe10 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.468+0000 7f7a24ff9700 1 -- 172.21.15.93:0/3781359079 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7a1c00de60 con 0x7f7a280ffe10 2023-12-14T18:25:51.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.468+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a2810f270 con 0x7f7a280ffe10 2023-12-14T18:25:51.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.468+0000 7f7a24ff9700 1 -- 172.21.15.93:0/3781359079 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a1c00f620 con 0x7f7a280ffe10 2023-12-14T18:25:51.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.468+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a2810f0e0 con 0x7f7a280ffe10 2023-12-14T18:25:51.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.469+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a2810f400 con 0x7f7a280ffe10 2023-12-14T18:25:51.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.470+0000 7f7a24ff9700 1 -- 172.21.15.93:0/3781359079 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f7a1c010460 con 0x7f7a280ffe10 2023-12-14T18:25:51.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.471+0000 7f7a24ff9700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a100a11e0 0x7f7a100a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:51.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.471+0000 7f7a277fe700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a100a11e0 0x7f7a100a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:51.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.471+0000 7f7a24ff9700 1 -- 172.21.15.93:0/3781359079 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7a1c0cc8e0 con 0x7f7a280ffe10 2023-12-14T18:25:51.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.472+0000 7f7a277fe700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a100a11e0 0x7f7a100a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005950 tx=0x7f7a180058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:51.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.475+0000 7f7a24ff9700 1 -- 172.21.15.93:0/3781359079 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7a1c095bc0 con 0x7f7a280ffe10 2023-12-14T18:25:51.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:51 smithi160 ceph-mon[43840]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2023-12-14T18:25:51.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:51 smithi160 ceph-mon[43840]: pgmap v263: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s; 90 B/s, 0 keys/s, 0 objects/s recovering 2023-12-14T18:25:51.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.708+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7a28066ea0 con 0x7f7a280ffe10 2023-12-14T18:25:51.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.708+0000 7f7a24ff9700 1 -- 172.21.15.93:0/3781359079 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7a1c0991e0 con 0x7f7a280ffe10 2023-12-14T18:25:51.709 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:51.709 INFO:teuthology.orchestra.run.smithi093.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-14T18:25:51.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a100a11e0 msgr2=0x7f7a100a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a100a11e0 0x7f7a100a3690 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7a18005950 tx=0x7f7a180058e0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a280ffe10 msgr2=0x7f7a28196a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:51.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a280ffe10 0x7f7a28196a70 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7a1c00d980 tx=0x7f7a1c00dd40 comp rx=0 tx=0).stop 2023-12-14T18:25:51.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 shutdown_connections 2023-12-14T18:25:51.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f7a100a11e0 0x7f7a100a3690 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7a28104510 0x7f7a2819aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7a280ff470 0x7f7a28196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 --2- 172.21.15.93:0/3781359079 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7a280ffe10 0x7f7a28196a70 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:51.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 >> 172.21.15.93:0/3781359079 conn(0x7f7a280770d0 msgr2=0x7f7a28068e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:51.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 shutdown_connections 2023-12-14T18:25:51.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:51.710+0000 7f7a2dadd700 1 -- 172.21.15.93:0/3781359079 wait complete. 2023-12-14T18:25:52.054 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2023-12-14T18:25:52.054 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 -- 172.21.15.93:0/3453389683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8168100350 msgr2=0x7f81681007a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 --2- 172.21.15.93:0/3453389683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8168100350 0x7f81681007a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f815c009990 tx=0x7f815c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 -- 172.21.15.93:0/3453389683 shutdown_connections 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 --2- 172.21.15.93:0/3453389683 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168100ce0 0x7f81681075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 --2- 172.21.15.93:0/3453389683 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f81680ff160 0x7f81680ff570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 --2- 172.21.15.93:0/3453389683 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8168100350 0x7f81681007a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 -- 172.21.15.93:0/3453389683 >> 172.21.15.93:0/3453389683 conn(0x7f81680fa740 msgr2=0x7f81680fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 -- 172.21.15.93:0/3453389683 shutdown_connections 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 -- 172.21.15.93:0/3453389683 wait complete. 2023-12-14T18:25:52.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.181+0000 7f816e251700 1 Processor -- start 2023-12-14T18:25:52.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f816e251700 1 -- start start 2023-12-14T18:25:52.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f816e251700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81680ff160 0x7f8168195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f816e251700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168100350 0x7f81681964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f816e251700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8168100ce0 0x7f816819d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f816e251700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819db10 con 0x7f81680ff160 2023-12-14T18:25:52.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f816e251700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819dc50 con 0x7f8168100350 2023-12-14T18:25:52.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f816e251700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f816819dde0 con 0x7f8168100ce0 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f81677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81680ff160 0x7f8168195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f81677fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81680ff160 0x7f8168195f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46884/0 (socket says 172.21.15.93:46884) 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f81677fe700 1 -- 172.21.15.93:0/3766984451 learned_addr learned my addr 172.21.15.93:0/3766984451 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f8166ffd700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168100350 0x7f81681964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f8167fff700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8168100ce0 0x7f816819d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f81677fe700 1 -- 172.21.15.93:0/3766984451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8168100ce0 msgr2=0x7f816819d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f81677fe700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8168100ce0 0x7f816819d540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f81677fe700 1 -- 172.21.15.93:0/3766984451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168100350 msgr2=0x7f81681964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f81677fe700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168100350 0x7f81681964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f81677fe700 1 -- 172.21.15.93:0/3766984451 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c0095f0 con 0x7f81680ff160 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f81677fe700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81680ff160 0x7f8168195f90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f815800d980 tx=0x7f815800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f8164ff9700 1 -- 172.21.15.93:0/3766984451 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158004020 con 0x7f81680ff160 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f8164ff9700 1 -- 172.21.15.93:0/3766984451 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8158009730 con 0x7f81680ff160 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.182+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f816810aac0 con 0x7f81680ff160 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.183+0000 7f8164ff9700 1 -- 172.21.15.93:0/3766984451 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8158003a80 con 0x7f81680ff160 2023-12-14T18:25:52.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.183+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f816810a930 con 0x7f81680ff160 2023-12-14T18:25:52.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.183+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816819e580 con 0x7f81680ff160 2023-12-14T18:25:52.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.184+0000 7f8164ff9700 1 -- 172.21.15.93:0/3766984451 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f81580098a0 con 0x7f81680ff160 2023-12-14T18:25:52.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.185+0000 7f8164ff9700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f81500a1260 0x7f81500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.185+0000 7f8166ffd700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f81500a1260 0x7f81500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.186 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.185+0000 7f8164ff9700 1 -- 172.21.15.93:0/3766984451 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f81580cca60 con 0x7f81680ff160 2023-12-14T18:25:52.187 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.185+0000 7f8166ffd700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f81500a1260 0x7f81500a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b5c0 tx=0x7f815c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.189+0000 7f8164ff9700 1 -- 172.21.15.93:0/3766984451 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8158095d40 con 0x7f81680ff160 2023-12-14T18:25:52.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:52 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3781359079' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-14T18:25:52.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:52 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3781359079' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-14T18:25:52.380 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.379+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f816819eb30 con 0x7f81680ff160 2023-12-14T18:25:52.382 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.381+0000 7f8164ff9700 1 -- 172.21.15.93:0/3766984451 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24357 (secure 0 0 0) 0x7f8158099360 con 0x7f81680ff160 2023-12-14T18:25:52.382 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:52.382 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":86,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:25:37.836200+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:25:52.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f81500a1260 msgr2=0x7f81500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f81500a1260 0x7f81500a3710 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f815c00b5c0 tx=0x7f815c005960 comp rx=0 tx=0).stop 2023-12-14T18:25:52.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81680ff160 msgr2=0x7f8168195f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81680ff160 0x7f8168195f90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f815800d980 tx=0x7f815800dc90 comp rx=0 tx=0).stop 2023-12-14T18:25:52.384 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 shutdown_connections 2023-12-14T18:25:52.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f81500a1260 0x7f81500a3710 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f81680ff160 0x7f8168195f90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8168100ce0 0x7f816819d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 --2- 172.21.15.93:0/3766984451 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8168100350 0x7f81681964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 >> 172.21.15.93:0/3766984451 conn(0x7f81680fa740 msgr2=0x7f8168068eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:52.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 shutdown_connections 2023-12-14T18:25:52.385 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.383+0000 7f816e251700 1 -- 172.21.15.93:0/3766984451 wait complete. 2023-12-14T18:25:52.397 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:24:34.407045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:24:35.728548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:24:38.796189+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:25:52.397 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:25:52.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:52 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3781359079' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2023-12-14T18:25:52.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 -- 172.21.15.93:0/4105874755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840feb40 msgr2=0x7fc384108720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4105874755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840feb40 0x7fc384108720 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fc37c00b0a0 tx=0x7fc37c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 -- 172.21.15.93:0/4105874755 shutdown_connections 2023-12-14T18:25:52.524 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4105874755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840feb40 0x7fc384108720 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4105874755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc384101740 0x7fc384101b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4105874755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3840fe1b0 0x7fc3840fe600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 -- 172.21.15.93:0/4105874755 >> 172.21.15.93:0/4105874755 conn(0x7fc3840f9de0 msgr2=0x7fc3840fc1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 -- 172.21.15.93:0/4105874755 shutdown_connections 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 -- 172.21.15.93:0/4105874755 wait complete. 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 Processor -- start 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.522+0000 7fc38aba0700 1 -- start start 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38aba0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840fe1b0 0x7fc384195f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38aba0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3840feb40 0x7fc384196490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38aba0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc384101740 0x7fc38419d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38aba0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38419dad0 con 0x7fc3840fe1b0 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38aba0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38419dc10 con 0x7fc3840feb40 2023-12-14T18:25:52.525 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38aba0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc38419dda0 con 0x7fc384101740 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc389b9e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840fe1b0 0x7fc384195f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc389b9e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840fe1b0 0x7fc384195f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46900/0 (socket says 172.21.15.93:46900) 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc389b9e700 1 -- 172.21.15.93:0/4150201991 learned_addr learned my addr 172.21.15.93:0/4150201991 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38939d700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3840feb40 0x7fc384196490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38a39f700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc384101740 0x7fc38419d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc389b9e700 1 -- 172.21.15.93:0/4150201991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc384101740 msgr2=0x7fc38419d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc389b9e700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc384101740 0x7fc38419d500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc389b9e700 1 -- 172.21.15.93:0/4150201991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3840feb40 msgr2=0x7fc384196490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc389b9e700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3840feb40 0x7fc384196490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc389b9e700 1 -- 172.21.15.93:0/4150201991 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc37c009d00 con 0x7fc3840fe1b0 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38939d700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3840feb40 0x7fc384196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc389b9e700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840fe1b0 0x7fc384195f50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc37400e970 tx=0x7fc37400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc37affd700 1 -- 172.21.15.93:0/4150201991 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374009790 con 0x7fc3840fe1b0 2023-12-14T18:25:52.526 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc37affd700 1 -- 172.21.15.93:0/4150201991 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3740044e0 con 0x7fc3840fe1b0 2023-12-14T18:25:52.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc38410ba80 con 0x7fc3840fe1b0 2023-12-14T18:25:52.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc37affd700 1 -- 172.21.15.93:0/4150201991 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc374003e30 con 0x7fc3840fe1b0 2023-12-14T18:25:52.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.523+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc38410b8f0 con 0x7fc3840fe1b0 2023-12-14T18:25:52.527 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.524+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc384107170 con 0x7fc3840fe1b0 2023-12-14T18:25:52.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.525+0000 7fc37affd700 1 -- 172.21.15.93:0/4150201991 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc374003f90 con 0x7fc3840fe1b0 2023-12-14T18:25:52.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.526+0000 7fc37affd700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3700a12e0 0x7fc3700a3790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.526+0000 7fc38939d700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3700a12e0 0x7fc3700a3790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.526+0000 7fc37affd700 1 -- 172.21.15.93:0/4150201991 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc374014070 con 0x7fc3840fe1b0 2023-12-14T18:25:52.528 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.526+0000 7fc38939d700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3700a12e0 0x7fc3700a3790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc380009be0 tx=0x7fc380009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.530+0000 7fc37affd700 1 -- 172.21.15.93:0/4150201991 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc374096330 con 0x7fc3840fe1b0 2023-12-14T18:25:52.722 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.722+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc3840619b0 con 0x7fc3840fe1b0 2023-12-14T18:25:52.723 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.722+0000 7fc37affd700 1 -- 172.21.15.93:0/4150201991 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc374016330 con 0x7fc3840fe1b0 2023-12-14T18:25:52.723 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:25:52.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3700a12e0 msgr2=0x7fc3700a3790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3700a12e0 0x7fc3700a3790 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc380009be0 tx=0x7fc380009260 comp rx=0 tx=0).stop 2023-12-14T18:25:52.725 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840fe1b0 msgr2=0x7fc384195f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840fe1b0 0x7fc384195f50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fc37400e970 tx=0x7fc37400ed30 comp rx=0 tx=0).stop 2023-12-14T18:25:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 shutdown_connections 2023-12-14T18:25:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3700a12e0 0x7fc3700a3790 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3840fe1b0 0x7fc384195f50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc384101740 0x7fc38419d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 --2- 172.21.15.93:0/4150201991 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3840feb40 0x7fc384196490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.724+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 >> 172.21.15.93:0/4150201991 conn(0x7fc3840f9de0 msgr2=0x7fc3840fbf90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.725+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 shutdown_connections 2023-12-14T18:25:52.726 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.725+0000 7fc38aba0700 1 -- 172.21.15.93:0/4150201991 wait complete. 2023-12-14T18:25:52.738 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:25:52.863 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 -- 172.21.15.93:0/3387054987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730100220 msgr2=0x7fa730100630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 --2- 172.21.15.93:0/3387054987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730100220 0x7fa730100630 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa724009990 tx=0x7fa724009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 -- 172.21.15.93:0/3387054987 shutdown_connections 2023-12-14T18:25:52.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 --2- 172.21.15.93:0/3387054987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730100220 0x7fa730100630 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 --2- 172.21.15.93:0/3387054987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730101da0 0x7fa730108660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 --2- 172.21.15.93:0/3387054987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa730101410 0x7fa730101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 -- 172.21.15.93:0/3387054987 >> 172.21.15.93:0/3387054987 conn(0x7fa7300fb840 msgr2=0x7fa7300fdc70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:52.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 -- 172.21.15.93:0/3387054987 shutdown_connections 2023-12-14T18:25:52.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 -- 172.21.15.93:0/3387054987 wait complete. 2023-12-14T18:25:52.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 Processor -- start 2023-12-14T18:25:52.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 -- start start 2023-12-14T18:25:52.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.863+0000 7fa737258700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730100220 0x7fa7300748a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa737258700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa730101410 0x7fa730072ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa737258700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730101da0 0x7fa730073420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa737258700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa730073bc0 con 0x7fa730101da0 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa737258700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa730073d30 con 0x7fa730101410 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa737258700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7301a0030 con 0x7fa730100220 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa7357f5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730101da0 0x7fa730073420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa7357f5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730101da0 0x7fa730073420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46914/0 (socket says 172.21.15.93:46914) 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa7357f5700 1 -- 172.21.15.93:0/881218309 learned_addr learned my addr 172.21.15.93:0/881218309 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa734ff4700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730100220 0x7fa7300748a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa72ffff700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa730101410 0x7fa730072ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa7357f5700 1 -- 172.21.15.93:0/881218309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730100220 msgr2=0x7fa7300748a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa7357f5700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730100220 0x7fa7300748a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa7357f5700 1 -- 172.21.15.93:0/881218309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa730101410 msgr2=0x7fa730072ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa7357f5700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa730101410 0x7fa730072ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa7357f5700 1 -- 172.21.15.93:0/881218309 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7240095f0 con 0x7fa730101da0 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa7357f5700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730101da0 0x7fa730073420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa728007ee0 tx=0x7fa72800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa72dffb700 1 -- 172.21.15.93:0/881218309 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728017070 con 0x7fa730101da0 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa72dffb700 1 -- 172.21.15.93:0/881218309 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa72800f040 con 0x7fa730101da0 2023-12-14T18:25:52.867 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa73010bb90 con 0x7fa730101da0 2023-12-14T18:25:52.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa72dffb700 1 -- 172.21.15.93:0/881218309 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa728005580 con 0x7fa730101da0 2023-12-14T18:25:52.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.864+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa73010ba00 con 0x7fa730101da0 2023-12-14T18:25:52.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.865+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa730107810 con 0x7fa730101da0 2023-12-14T18:25:52.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.866+0000 7fa72dffb700 1 -- 172.21.15.93:0/881218309 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa72800db20 con 0x7fa730101da0 2023-12-14T18:25:52.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.867+0000 7fa72dffb700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7200a11e0 0x7fa7200a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:52.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.867+0000 7fa734ff4700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7200a11e0 0x7fa7200a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:52.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.867+0000 7fa72dffb700 1 -- 172.21.15.93:0/881218309 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa7280cc540 con 0x7fa730101da0 2023-12-14T18:25:52.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.867+0000 7fa734ff4700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7200a11e0 0x7fa7200a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa72400b5c0 tx=0x7fa724005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:52.872 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:52.871+0000 7fa72dffb700 1 -- 172.21.15.93:0/881218309 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa7280957c0 con 0x7fa730101da0 2023-12-14T18:25:53.060 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.060+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa730061700 con 0x7fa730101da0 2023-12-14T18:25:53.061 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.060+0000 7fa72dffb700 1 -- 172.21.15.93:0/881218309 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa728098de0 con 0x7fa730101da0 2023-12-14T18:25:53.061 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:25:53.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7200a11e0 msgr2=0x7fa7200a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7200a11e0 0x7fa7200a3690 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa72400b5c0 tx=0x7fa724005960 comp rx=0 tx=0).stop 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730101da0 msgr2=0x7fa730073420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730101da0 0x7fa730073420 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa728007ee0 tx=0x7fa72800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 shutdown_connections 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7200a11e0 0x7fa7200a3690 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa730101da0 0x7fa730073420 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa730100220 0x7fa7300748a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 --2- 172.21.15.93:0/881218309 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa730101410 0x7fa730072ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.062+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 >> 172.21.15.93:0/881218309 conn(0x7fa7300fb840 msgr2=0x7fa730104950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.063+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 shutdown_connections 2023-12-14T18:25:53.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.063+0000 7fa737258700 1 -- 172.21.15.93:0/881218309 wait complete. 2023-12-14T18:25:53.076 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:25:53.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.199+0000 7f56358cb700 1 -- 172.21.15.93:0/3413014459 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56300ff1a0 msgr2=0x7f56300ff5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.199+0000 7f56358cb700 1 --2- 172.21.15.93:0/3413014459 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56300ff1a0 0x7f56300ff5b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f5620009990 tx=0x7f5620009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.200 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.199+0000 7f56358cb700 1 -- 172.21.15.93:0/3413014459 shutdown_connections 2023-12-14T18:25:53.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.199+0000 7f56358cb700 1 --2- 172.21.15.93:0/3413014459 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56300ff1a0 0x7f56300ff5b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.199+0000 7f56358cb700 1 --2- 172.21.15.93:0/3413014459 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5630100d20 0x7f56301075e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.199+0000 7f56358cb700 1 --2- 172.21.15.93:0/3413014459 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5630100390 0x7f56301007e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.199+0000 7f56358cb700 1 -- 172.21.15.93:0/3413014459 >> 172.21.15.93:0/3413014459 conn(0x7f56300fa7c0 msgr2=0x7f56300fcbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:53.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.199+0000 7f56358cb700 1 -- 172.21.15.93:0/3413014459 shutdown_connections 2023-12-14T18:25:53.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.199+0000 7f56358cb700 1 -- 172.21.15.93:0/3413014459 wait complete. 2023-12-14T18:25:53.201 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f56358cb700 1 Processor -- start 2023-12-14T18:25:53.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f56358cb700 1 -- start start 2023-12-14T18:25:53.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f56358cb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56300ff1a0 0x7f5630195f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f56358cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5630100390 0x7f56301964d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f56358cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5630100d20 0x7f563019d540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f56358cb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019db10 con 0x7f5630100d20 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f56358cb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019dc50 con 0x7f5630100390 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f56358cb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f563019dde0 con 0x7f56300ff1a0 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f562f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5630100d20 0x7f563019d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f562f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5630100d20 0x7f563019d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46920/0 (socket says 172.21.15.93:46920) 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f562f7fe700 1 -- 172.21.15.93:0/1464983414 learned_addr learned my addr 172.21.15.93:0/1464983414 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f562e7fc700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5630100390 0x7f56301964d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.200+0000 7f562effd700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56300ff1a0 0x7f5630195f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f562f7fe700 1 -- 172.21.15.93:0/1464983414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56300ff1a0 msgr2=0x7f5630195f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f562f7fe700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56300ff1a0 0x7f5630195f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f562f7fe700 1 -- 172.21.15.93:0/1464983414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5630100390 msgr2=0x7f56301964d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f562f7fe700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5630100390 0x7f56301964d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f562f7fe700 1 -- 172.21.15.93:0/1464983414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56200095f0 con 0x7f5630100d20 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f562f7fe700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5630100d20 0x7f563019d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f562800ef30 tx=0x7f562800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:53.204 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f56348c9700 1 -- 172.21.15.93:0/1464983414 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5628018070 con 0x7f5630100d20 2023-12-14T18:25:53.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f56348c9700 1 -- 172.21.15.93:0/1464983414 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f562800f040 con 0x7f5630100d20 2023-12-14T18:25:53.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f563010aaa0 con 0x7f5630100d20 2023-12-14T18:25:53.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f56348c9700 1 -- 172.21.15.93:0/1464983414 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f562800cd00 con 0x7f5630100d20 2023-12-14T18:25:53.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.201+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f563010a910 con 0x7f5630100d20 2023-12-14T18:25:53.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.202+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f563019e420 con 0x7f5630100d20 2023-12-14T18:25:53.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.204+0000 7f56348c9700 1 -- 172.21.15.93:0/1464983414 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5628003a00 con 0x7f5630100d20 2023-12-14T18:25:53.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.204+0000 7f56348c9700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f56180a11e0 0x7f56180a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.204+0000 7f562effd700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f56180a11e0 0x7f56180a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.205+0000 7f56348c9700 1 -- 172.21.15.93:0/1464983414 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f56280cd310 con 0x7f5630100d20 2023-12-14T18:25:53.206 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.205+0000 7f562effd700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f56180a11e0 0x7f56180a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5620006010 tx=0x7f5620005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:53.208 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.208+0000 7f56348c9700 1 -- 172.21.15.93:0/1464983414 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5628096620 con 0x7f5630100d20 2023-12-14T18:25:53.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:53 smithi093 ceph-mon[42070]: pgmap v264: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 90 B/s, 0 objects/s recovering 2023-12-14T18:25:53.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:53 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3766984451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:53.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:53 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4150201991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:53.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:53 smithi097 ceph-mon[43486]: pgmap v264: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 90 B/s, 0 objects/s recovering 2023-12-14T18:25:53.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:53 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3766984451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:53.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:53 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4150201991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:53.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.398+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f563019e9d0 con 0x7f5630100d20 2023-12-14T18:25:53.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.398+0000 7f56348c9700 1 -- 172.21.15.93:0/1464983414 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5628099c40 con 0x7f5630100d20 2023-12-14T18:25:53.399 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-12-14T18:25:53.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f56180a11e0 msgr2=0x7f56180a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f56180a11e0 0x7f56180a3690 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5620006010 tx=0x7f5620005960 comp rx=0 tx=0).stop 2023-12-14T18:25:53.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5630100d20 msgr2=0x7f563019d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5630100d20 0x7f563019d540 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f562800ef30 tx=0x7f562800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 shutdown_connections 2023-12-14T18:25:53.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f56180a11e0 0x7f56180a3690 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5630100d20 0x7f563019d540 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56300ff1a0 0x7f5630195f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 --2- 172.21.15.93:0/1464983414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5630100390 0x7f56301964d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 >> 172.21.15.93:0/1464983414 conn(0x7f56300fa7c0 msgr2=0x7f56300fb080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:53.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 shutdown_connections 2023-12-14T18:25:53.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.400+0000 7f56358cb700 1 -- 172.21.15.93:0/1464983414 wait complete. 2023-12-14T18:25:53.413 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:25:53.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:53 smithi160 ceph-mon[43840]: pgmap v264: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s; 90 B/s, 0 objects/s recovering 2023-12-14T18:25:53.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:53 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3766984451' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:53.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:53 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4150201991' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:53.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 -- 172.21.15.93:0/1133051889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 msgr2=0x7fa7640ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/1133051889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 0x7fa7640ff530 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa760009990 tx=0x7fa760009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 -- 172.21.15.93:0/1133051889 shutdown_connections 2023-12-14T18:25:53.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/1133051889 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 0x7fa7640ff530 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/1133051889 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa764100ca0 0x7fa764107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/1133051889 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa764100310 0x7fa764100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 -- 172.21.15.93:0/1133051889 >> 172.21.15.93:0/1133051889 conn(0x7fa7640fa740 msgr2=0x7fa7640fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 -- 172.21.15.93:0/1133051889 shutdown_connections 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 -- 172.21.15.93:0/1133051889 wait complete. 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 Processor -- start 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 -- start start 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 0x7fa764194f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa764100310 0x7fa7641954c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa764100ca0 0x7fa76419c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76419cb00 con 0x7fa7640ff120 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76419cc40 con 0x7fa764100310 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.538+0000 7fa76b9dd700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa76419cdd0 con 0x7fa764100ca0 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769779700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 0x7fa764194f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769779700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 0x7fa764194f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46942/0 (socket says 172.21.15.93:46942) 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769779700 1 -- 172.21.15.93:0/156898411 learned_addr learned my addr 172.21.15.93:0/156898411 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa768f78700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa764100310 0x7fa7641954c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.541 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769f7a700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa764100ca0 0x7fa76419c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769779700 1 -- 172.21.15.93:0/156898411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa764100ca0 msgr2=0x7fa76419c530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769779700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa764100ca0 0x7fa76419c530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769779700 1 -- 172.21.15.93:0/156898411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa764100310 msgr2=0x7fa7641954c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769779700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa764100310 0x7fa7641954c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769779700 1 -- 172.21.15.93:0/156898411 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7600095f0 con 0x7fa7640ff120 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa769779700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 0x7fa764194f80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa760004730 tx=0x7fa760004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa75a7fc700 1 -- 172.21.15.93:0/156898411 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa76001d070 con 0x7fa7640ff120 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa75a7fc700 1 -- 172.21.15.93:0/156898411 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa760021410 con 0x7fa7640ff120 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa75a7fc700 1 -- 172.21.15.93:0/156898411 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa760017650 con 0x7fa7640ff120 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa76410aae0 con 0x7fa7640ff120 2023-12-14T18:25:53.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.539+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa76410a950 con 0x7fa7640ff120 2023-12-14T18:25:53.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.540+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa764106760 con 0x7fa7640ff120 2023-12-14T18:25:53.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.541+0000 7fa75a7fc700 1 -- 172.21.15.93:0/156898411 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa760021580 con 0x7fa7640ff120 2023-12-14T18:25:53.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.542+0000 7fa75a7fc700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7500a1260 0x7fa7500a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.542+0000 7fa768f78700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7500a1260 0x7fa7500a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.542+0000 7fa75a7fc700 1 -- 172.21.15.93:0/156898411 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa7600ce110 con 0x7fa7640ff120 2023-12-14T18:25:53.543 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.542+0000 7fa768f78700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7500a1260 0x7fa7500a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa754009c60 tx=0x7fa754009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:53.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.546+0000 7fa75a7fc700 1 -- 172.21.15.93:0/156898411 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa760097480 con 0x7fa7640ff120 2023-12-14T18:25:53.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.733+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa764061930 con 0x7fa7640ff120 2023-12-14T18:25:53.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.734+0000 7fa75a7fc700 1 -- 172.21.15.93:0/156898411 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa76009aaa0 con 0x7fa7640ff120 2023-12-14T18:25:53.735 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7500a1260 msgr2=0x7fa7500a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7500a1260 0x7fa7500a3710 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fa754009c60 tx=0x7fa754009260 comp rx=0 tx=0).stop 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 msgr2=0x7fa764194f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 0x7fa764194f80 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fa760004730 tx=0x7fa760004760 comp rx=0 tx=0).stop 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 shutdown_connections 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa7500a1260 0x7fa7500a3710 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa7640ff120 0x7fa764194f80 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa764100ca0 0x7fa76419c530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 --2- 172.21.15.93:0/156898411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa764100310 0x7fa7641954c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 >> 172.21.15.93:0/156898411 conn(0x7fa7640fa740 msgr2=0x7fa7641038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:53.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 shutdown_connections 2023-12-14T18:25:53.738 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.736+0000 7fa76b9dd700 1 -- 172.21.15.93:0/156898411 wait complete. 2023-12-14T18:25:53.749 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:53.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.870+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3492059433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1841035c0 msgr2=0x7fb1841059a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.870+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3492059433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1841035c0 0x7fb1841059a0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fb178009990 tx=0x7fb178009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.870+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3492059433 shutdown_connections 2023-12-14T18:25:53.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.870+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3492059433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb184105ee0 0x7fb184108340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.870+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3492059433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184100ca0 0x7fb184103080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.870+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3492059433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1841035c0 0x7fb1841059a0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.870+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3492059433 >> 172.21.15.93:0/3492059433 conn(0x7fb1840fa700 msgr2=0x7fb1840fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:53.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.870+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3492059433 shutdown_connections 2023-12-14T18:25:53.871 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.870+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3492059433 wait complete. 2023-12-14T18:25:53.873 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb18a2c4700 1 Processor -- start 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb18a2c4700 1 -- start start 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb18a2c4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184100ca0 0x7fb184195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb18a2c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1841035c0 0x7fb1841964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb18a2c4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb184105ee0 0x7fb18419d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb18a2c4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18419db00 con 0x7fb184105ee0 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb18a2c4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18419dc40 con 0x7fb184100ca0 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb18a2c4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb18419ddd0 con 0x7fb1841035c0 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb188861700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb184105ee0 0x7fb18419d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb183fff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184100ca0 0x7fb184195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb188861700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb184105ee0 0x7fb18419d530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46956/0 (socket says 172.21.15.93:46956) 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb1837fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1841035c0 0x7fb1841964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.871+0000 7fb188861700 1 -- 172.21.15.93:0/3447375235 learned_addr learned my addr 172.21.15.93:0/3447375235 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb188861700 1 -- 172.21.15.93:0/3447375235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1841035c0 msgr2=0x7fb1841964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.874 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb188861700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1841035c0 0x7fb1841964c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb188861700 1 -- 172.21.15.93:0/3447375235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184100ca0 msgr2=0x7fb184195f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb188861700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184100ca0 0x7fb184195f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb188861700 1 -- 172.21.15.93:0/3447375235 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1780095f0 con 0x7fb184105ee0 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb188861700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb184105ee0 0x7fb18419d530 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00ada0 tx=0x7fb17c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb1817fa700 1 -- 172.21.15.93:0/3447375235 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c00f070 con 0x7fb184105ee0 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb1817fa700 1 -- 172.21.15.93:0/3447375235 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb17c007b30 con 0x7fb184105ee0 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb1817fa700 1 -- 172.21.15.93:0/3447375235 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb17c005280 con 0x7fb184105ee0 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb18410bba0 con 0x7fb184105ee0 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.872+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb18410ba10 con 0x7fb184105ee0 2023-12-14T18:25:53.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.873+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb184067160 con 0x7fb184105ee0 2023-12-14T18:25:53.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.874+0000 7fb1817fa700 1 -- 172.21.15.93:0/3447375235 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fb17c00a090 con 0x7fb184105ee0 2023-12-14T18:25:53.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.874+0000 7fb1817fa700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb16c0a11e0 0x7fb16c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:53.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.875+0000 7fb183fff700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb16c0a11e0 0x7fb16c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:53.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.875+0000 7fb1817fa700 1 -- 172.21.15.93:0/3447375235 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb17c0cc8e0 con 0x7fb184105ee0 2023-12-14T18:25:53.876 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.875+0000 7fb183fff700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb16c0a11e0 0x7fb16c0a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb174005950 tx=0x7fb17400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:53.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:53.878+0000 7fb1817fa700 1 -- 172.21.15.93:0/3447375235 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb17c099680 con 0x7fb184105ee0 2023-12-14T18:25:54.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.115+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb18419e890 con 0x7fb184105ee0 2023-12-14T18:25:54.116 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.115+0000 7fb1817fa700 1 -- 172.21.15.93:0/3447375235 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10799 (secure 0 0 0) 0x7fb17c099210 con 0x7fb184105ee0 2023-12-14T18:25:54.116 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:54.117 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:25:01.760987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:25:54.118 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.117+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb16c0a11e0 msgr2=0x7fb16c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.117+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb16c0a11e0 0x7fb16c0a3690 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb174005950 tx=0x7fb17400b410 comp rx=0 tx=0).stop 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.117+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb184105ee0 msgr2=0x7fb18419d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.117+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb184105ee0 0x7fb18419d530 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fb17c00ada0 tx=0x7fb17c00c330 comp rx=0 tx=0).stop 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.118+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 shutdown_connections 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.118+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb16c0a11e0 0x7fb16c0a3690 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.118+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb184105ee0 0x7fb18419d530 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.118+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb184100ca0 0x7fb184195f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.118+0000 7fb18a2c4700 1 --2- 172.21.15.93:0/3447375235 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1841035c0 0x7fb1841964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.118+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 >> 172.21.15.93:0/3447375235 conn(0x7fb1840fa700 msgr2=0x7fb1841020e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.118+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 shutdown_connections 2023-12-14T18:25:54.119 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.118+0000 7fb18a2c4700 1 -- 172.21.15.93:0/3447375235 wait complete. 2023-12-14T18:25:54.120 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 24 2023-12-14T18:25:54.131 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:25:54.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 -- 172.21.15.93:0/1370554463 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98100250 msgr2=0x7fba98100660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 --2- 172.21.15.93:0/1370554463 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98100250 0x7fba98100660 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fba88009990 tx=0x7fba88009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 -- 172.21.15.93:0/1370554463 shutdown_connections 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 --2- 172.21.15.93:0/1370554463 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98100250 0x7fba98100660 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 --2- 172.21.15.93:0/1370554463 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba98101dd0 0x7fba98108690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 --2- 172.21.15.93:0/1370554463 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba98101440 0x7fba98101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 -- 172.21.15.93:0/1370554463 >> 172.21.15.93:0/1370554463 conn(0x7fba980fb850 msgr2=0x7fba980fdca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 -- 172.21.15.93:0/1370554463 shutdown_connections 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 -- 172.21.15.93:0/1370554463 wait complete. 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.252+0000 7fba9ccad700 1 Processor -- start 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9ccad700 1 -- start start 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9ccad700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba98100250 0x7fba98196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9ccad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba98101440 0x7fba98196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9ccad700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98101dd0 0x7fba9819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9ccad700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9819db90 con 0x7fba98101dd0 2023-12-14T18:25:54.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9ccad700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9819dcd0 con 0x7fba98101440 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9ccad700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9819de60 con 0x7fba98100250 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba96d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98101dd0 0x7fba9819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba96d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98101dd0 0x7fba9819d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46976/0 (socket says 172.21.15.93:46976) 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba96d9d700 1 -- 172.21.15.93:0/573111674 learned_addr learned my addr 172.21.15.93:0/573111674 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba95d9b700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba98101440 0x7fba98196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9659c700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba98100250 0x7fba98196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba96d9d700 1 -- 172.21.15.93:0/573111674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba98100250 msgr2=0x7fba98196010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba96d9d700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba98100250 0x7fba98196010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba96d9d700 1 -- 172.21.15.93:0/573111674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba98101440 msgr2=0x7fba98196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba96d9d700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba98101440 0x7fba98196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba96d9d700 1 -- 172.21.15.93:0/573111674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba880095f0 con 0x7fba98101dd0 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9659c700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba98100250 0x7fba98196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba96d9d700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98101dd0 0x7fba9819d5c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fba9000ada0 tx=0x7fba9000c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba877fe700 1 -- 172.21.15.93:0/573111674 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9000f070 con 0x7fba98101dd0 2023-12-14T18:25:54.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9810bc00 con 0x7fba98101dd0 2023-12-14T18:25:54.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba877fe700 1 -- 172.21.15.93:0/573111674 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fba90007b30 con 0x7fba98101dd0 2023-12-14T18:25:54.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.253+0000 7fba877fe700 1 -- 172.21.15.93:0/573111674 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900053e0 con 0x7fba98101dd0 2023-12-14T18:25:54.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.254+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9810ba70 con 0x7fba98101dd0 2023-12-14T18:25:54.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.255+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba981071d0 con 0x7fba98101dd0 2023-12-14T18:25:54.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.256+0000 7fba877fe700 1 -- 172.21.15.93:0/573111674 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fba9000c980 con 0x7fba98101dd0 2023-12-14T18:25:54.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.256+0000 7fba877fe700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fba800a11e0 0x7fba800a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.256+0000 7fba9659c700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fba800a11e0 0x7fba800a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.257+0000 7fba877fe700 1 -- 172.21.15.93:0/573111674 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fba900cc9e0 con 0x7fba98101dd0 2023-12-14T18:25:54.258 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.257+0000 7fba9659c700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fba800a11e0 0x7fba800a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba8800b5c0 tx=0x7fba88005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:54.261 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.260+0000 7fba877fe700 1 -- 172.21.15.93:0/573111674 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fba90095d50 con 0x7fba98101dd0 2023-12-14T18:25:54.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:54 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/881218309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:25:54.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:54 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1464983414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:25:54.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:54 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/156898411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:25:54.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:54 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/881218309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:25:54.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:54 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1464983414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:25:54.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:54 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/156898411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:25:54.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.453+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fba9802cfe0 con 0x7fba98101dd0 2023-12-14T18:25:54.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.455+0000 7fba877fe700 1 -- 172.21.15.93:0/573111674 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24357 (secure 0 0 0) 0x7fba90099370 con 0x7fba98101dd0 2023-12-14T18:25:54.456 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:54.456 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":86,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:25:37.836200+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:25:54.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fba800a11e0 msgr2=0x7fba800a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fba800a11e0 0x7fba800a3690 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fba8800b5c0 tx=0x7fba88005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98101dd0 msgr2=0x7fba9819d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98101dd0 0x7fba9819d5c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fba9000ada0 tx=0x7fba9000c330 comp rx=0 tx=0).stop 2023-12-14T18:25:54.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 shutdown_connections 2023-12-14T18:25:54.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fba800a11e0 0x7fba800a3690 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fba98101dd0 0x7fba9819d5c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.458 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fba98100250 0x7fba98196010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 --2- 172.21.15.93:0/573111674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fba98101440 0x7fba98196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 >> 172.21.15.93:0/573111674 conn(0x7fba980fb850 msgr2=0x7fba980fdb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:54.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 shutdown_connections 2023-12-14T18:25:54.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.457+0000 7fba9ccad700 1 -- 172.21.15.93:0/573111674 wait complete. 2023-12-14T18:25:54.470 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:24:34.407045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:24:35.728548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:24:38.796189+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:25:54.471 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:25:54.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:54 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/881218309' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:25:54.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:54 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1464983414' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:25:54.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:54 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/156898411' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:25:54.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.595+0000 7fe952db2700 1 -- 172.21.15.93:0/138550151 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c105f80 msgr2=0x7fe94c108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.595+0000 7fe952db2700 1 --2- 172.21.15.93:0/138550151 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c105f80 0x7fe94c108380 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fe94400b0a0 tx=0x7fe94400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.595+0000 7fe952db2700 1 -- 172.21.15.93:0/138550151 shutdown_connections 2023-12-14T18:25:54.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.595+0000 7fe952db2700 1 --2- 172.21.15.93:0/138550151 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c105f80 0x7fe94c108380 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.595+0000 7fe952db2700 1 --2- 172.21.15.93:0/138550151 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94c100d40 0x7fe94c103120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.598 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.595+0000 7fe952db2700 1 --2- 172.21.15.93:0/138550151 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe94c103660 0x7fe94c105a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.595+0000 7fe952db2700 1 -- 172.21.15.93:0/138550151 >> 172.21.15.93:0/138550151 conn(0x7fe94c0fa780 msgr2=0x7fe94c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.595+0000 7fe952db2700 1 -- 172.21.15.93:0/138550151 shutdown_connections 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.595+0000 7fe952db2700 1 -- 172.21.15.93:0/138550151 wait complete. 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe952db2700 1 Processor -- start 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe952db2700 1 -- start start 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe952db2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe94c100d40 0x7fe94c196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe952db2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c103660 0x7fe94c1965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe952db2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94c105f80 0x7fe94c19d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe952db2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c19dc00 con 0x7fe94c103660 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe952db2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c19dd40 con 0x7fe94c105f80 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe952db2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe94c19ded0 con 0x7fe94c100d40 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe94bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c103660 0x7fe94c1965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.599 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe94bfff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c103660 0x7fe94c1965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46992/0 (socket says 172.21.15.93:46992) 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe94bfff700 1 -- 172.21.15.93:0/1950024664 learned_addr learned my addr 172.21.15.93:0/1950024664 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe95134f700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94c105f80 0x7fe94c19d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe950b4e700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe94c100d40 0x7fe94c196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe94bfff700 1 -- 172.21.15.93:0/1950024664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe94c100d40 msgr2=0x7fe94c196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe94bfff700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe94c100d40 0x7fe94c196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe94bfff700 1 -- 172.21.15.93:0/1950024664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94c105f80 msgr2=0x7fe94c19d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe94bfff700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94c105f80 0x7fe94c19d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe94bfff700 1 -- 172.21.15.93:0/1950024664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe944009d00 con 0x7fe94c103660 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.596+0000 7fe94bfff700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c103660 0x7fe94c1965c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe93800e9a0 tx=0x7fe93800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.597+0000 7fe949ffb700 1 -- 172.21.15.93:0/1950024664 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938009790 con 0x7fe94c103660 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.597+0000 7fe949ffb700 1 -- 172.21.15.93:0/1950024664 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe93800cbd0 con 0x7fe94c103660 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.597+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe94c10bbc0 con 0x7fe94c103660 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.597+0000 7fe949ffb700 1 -- 172.21.15.93:0/1950024664 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe938018680 con 0x7fe94c103660 2023-12-14T18:25:54.600 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.597+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe94c10ba30 con 0x7fe94c103660 2023-12-14T18:25:54.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.598+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe94c0fc2f0 con 0x7fe94c103660 2023-12-14T18:25:54.601 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.599+0000 7fe949ffb700 1 -- 172.21.15.93:0/1950024664 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe93800cd40 con 0x7fe94c103660 2023-12-14T18:25:54.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.599+0000 7fe949ffb700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe93c0a11e0 0x7fe93c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.600+0000 7fe950b4e700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe93c0a11e0 0x7fe93c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.600+0000 7fe949ffb700 1 -- 172.21.15.93:0/1950024664 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe938014070 con 0x7fe94c103660 2023-12-14T18:25:54.602 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.600+0000 7fe950b4e700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe93c0a11e0 0x7fe93c0a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe94000bbd0 tx=0x7fe94000b5b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:54.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.603+0000 7fe949ffb700 1 -- 172.21.15.93:0/1950024664 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe938096560 con 0x7fe94c103660 2023-12-14T18:25:54.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.792+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe94c061930 con 0x7fe94c103660 2023-12-14T18:25:54.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.793+0000 7fe949ffb700 1 -- 172.21.15.93:0/1950024664 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe938099b80 con 0x7fe94c103660 2023-12-14T18:25:54.794 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:25:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe93c0a11e0 msgr2=0x7fe93c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe93c0a11e0 0x7fe93c0a3690 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fe94000bbd0 tx=0x7fe94000b5b0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c103660 msgr2=0x7fe94c1965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c103660 0x7fe94c1965c0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fe93800e9a0 tx=0x7fe93800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 shutdown_connections 2023-12-14T18:25:54.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe93c0a11e0 0x7fe93c0a3690 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe94c105f80 0x7fe94c19d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe94c100d40 0x7fe94c196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 --2- 172.21.15.93:0/1950024664 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe94c103660 0x7fe94c1965c0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 >> 172.21.15.93:0/1950024664 conn(0x7fe94c0fa780 msgr2=0x7fe94c1019a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 shutdown_connections 2023-12-14T18:25:54.797 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.795+0000 7fe952db2700 1 -- 172.21.15.93:0/1950024664 wait complete. 2023-12-14T18:25:54.808 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:25:54.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.932+0000 7f7068f52700 1 -- 172.21.15.93:0/447375912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7064101e30 msgr2=0x7f70641086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.935 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.932+0000 7f7068f52700 1 --2- 172.21.15.93:0/447375912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7064101e30 0x7f70641086f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f705c00b0a0 tx=0x7f705c00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.932+0000 7f7068f52700 1 -- 172.21.15.93:0/447375912 shutdown_connections 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.932+0000 7f7068f52700 1 --2- 172.21.15.93:0/447375912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7064101e30 0x7f70641086f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.932+0000 7f7068f52700 1 --2- 172.21.15.93:0/447375912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70640fe4d0 0x7f70640fe8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.932+0000 7f7068f52700 1 --2- 172.21.15.93:0/447375912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f70641014a0 0x7f70641018f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.932+0000 7f7068f52700 1 -- 172.21.15.93:0/447375912 >> 172.21.15.93:0/447375912 conn(0x7f70640f9ed0 msgr2=0x7f70640fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.932+0000 7f7068f52700 1 -- 172.21.15.93:0/447375912 shutdown_connections 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.932+0000 7f7068f52700 1 -- 172.21.15.93:0/447375912 wait complete. 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f7068f52700 1 Processor -- start 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f7068f52700 1 -- start start 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f7068f52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70640fe4d0 0x7f7064100e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f7068f52700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70641014a0 0x7f70640ff450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f7068f52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7064101e30 0x7f70640ff990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f7068f52700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7064100130 con 0x7f70640fe4d0 2023-12-14T18:25:54.936 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f7068f52700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70641002a0 con 0x7f70641014a0 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f7068f52700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70641a00e0 con 0x7f7064101e30 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f706259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70640fe4d0 0x7f7064100e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f706259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70640fe4d0 0x7f7064100e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47000/0 (socket says 172.21.15.93:47000) 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.933+0000 7f706259c700 1 -- 172.21.15.93:0/3854766733 learned_addr learned my addr 172.21.15.93:0/3854766733 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f7061d9b700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70641014a0 0x7f70640ff450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f7062d9d700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7064101e30 0x7f70640ff990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f706259c700 1 -- 172.21.15.93:0/3854766733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7064101e30 msgr2=0x7f70640ff990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f706259c700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7064101e30 0x7f70640ff990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f706259c700 1 -- 172.21.15.93:0/3854766733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70641014a0 msgr2=0x7f70640ff450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f706259c700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70641014a0 0x7f70640ff450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f706259c700 1 -- 172.21.15.93:0/3854766733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f705c009d00 con 0x7f70640fe4d0 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f706259c700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70640fe4d0 0x7f7064100e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f705800e970 tx=0x7f705800ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f70537fe700 1 -- 172.21.15.93:0/3854766733 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058009790 con 0x7f70640fe4d0 2023-12-14T18:25:54.937 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f70537fe700 1 -- 172.21.15.93:0/3854766733 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f705800c7e0 con 0x7f70640fe4d0 2023-12-14T18:25:54.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f706410bb80 con 0x7f70640fe4d0 2023-12-14T18:25:54.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f70537fe700 1 -- 172.21.15.93:0/3854766733 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7058003c70 con 0x7f70640fe4d0 2023-12-14T18:25:54.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.934+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f706410b9f0 con 0x7f70640fe4d0 2023-12-14T18:25:54.938 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.935+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70641071e0 con 0x7f70640fe4d0 2023-12-14T18:25:54.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.936+0000 7f70537fe700 1 -- 172.21.15.93:0/3854766733 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f705800cc50 con 0x7f70640fe4d0 2023-12-14T18:25:54.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.936+0000 7f70537fe700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f704c0a11e0 0x7f704c0a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:54.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.937+0000 7f70537fe700 1 -- 172.21.15.93:0/3854766733 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f7058014070 con 0x7f70640fe4d0 2023-12-14T18:25:54.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.937+0000 7f7061d9b700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f704c0a11e0 0x7f704c0a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:54.939 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.937+0000 7f7061d9b700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f704c0a11e0 0x7f704c0a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7054009c60 tx=0x7f7054009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:54.942 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:54.941+0000 7f70537fe700 1 -- 172.21.15.93:0/3854766733 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7058096240 con 0x7f70640fe4d0 2023-12-14T18:25:55.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:55 smithi093 ceph-mon[42070]: pgmap v265: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 80 B/s, 0 objects/s recovering 2023-12-14T18:25:55.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:55 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3447375235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:55.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:55 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/573111674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:55.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:55 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1950024664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:55.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.134+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f70640619b0 con 0x7f70640fe4d0 2023-12-14T18:25:55.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.135+0000 7f70537fe700 1 -- 172.21.15.93:0/3854766733 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7058003dd0 con 0x7f70640fe4d0 2023-12-14T18:25:55.136 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:25:55.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f704c0a11e0 msgr2=0x7f704c0a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f704c0a11e0 0x7f704c0a3690 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7054009c60 tx=0x7f7054009260 comp rx=0 tx=0).stop 2023-12-14T18:25:55.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70640fe4d0 msgr2=0x7f7064100e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70640fe4d0 0x7f7064100e10 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f705800e970 tx=0x7f705800ec80 comp rx=0 tx=0).stop 2023-12-14T18:25:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 shutdown_connections 2023-12-14T18:25:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f704c0a11e0 0x7f704c0a3690 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f70640fe4d0 0x7f7064100e10 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7064101e30 0x7f70640ff990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 --2- 172.21.15.93:0/3854766733 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f70641014a0 0x7f70640ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 >> 172.21.15.93:0/3854766733 conn(0x7f70640f9ed0 msgr2=0x7f70640fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 shutdown_connections 2023-12-14T18:25:55.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.137+0000 7f7068f52700 1 -- 172.21.15.93:0/3854766733 wait complete. 2023-12-14T18:25:55.150 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:25:55.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 -- 172.21.15.93:0/306979197 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 msgr2=0x7f32281087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 --2- 172.21.15.93:0/306979197 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 0x7f32281087b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f322000b0a0 tx=0x7f322000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 -- 172.21.15.93:0/306979197 shutdown_connections 2023-12-14T18:25:55.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 --2- 172.21.15.93:0/306979197 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 0x7f32281087b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 --2- 172.21.15.93:0/306979197 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280fe4f0 0x7f32280fe900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 --2- 172.21.15.93:0/306979197 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3228101560 0x7f32281019b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.277 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 -- 172.21.15.93:0/306979197 >> 172.21.15.93:0/306979197 conn(0x7f32280f9ed0 msgr2=0x7f32280fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 -- 172.21.15.93:0/306979197 shutdown_connections 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 -- 172.21.15.93:0/306979197 wait complete. 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.274+0000 7f322fa70700 1 Processor -- start 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322fa70700 1 -- start start 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322fa70700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280fe4f0 0x7f3228100ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322fa70700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3228101560 0x7f32280ff510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322fa70700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 0x7f32280ffa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322fa70700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32281001f0 con 0x7f3228101ef0 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322fa70700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3228100360 con 0x7f32280fe4f0 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322fa70700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32281a0140 con 0x7f3228101560 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322e00d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 0x7f32280ffa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.278 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322d80c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280fe4f0 0x7f3228100ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322e00d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 0x7f32280ffa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47012/0 (socket says 172.21.15.93:47012) 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322e00d700 1 -- 172.21.15.93:0/2745194852 learned_addr learned my addr 172.21.15.93:0/2745194852 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322d00b700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3228101560 0x7f32280ff510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322e00d700 1 -- 172.21.15.93:0/2745194852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3228101560 msgr2=0x7f32280ff510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322e00d700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3228101560 0x7f32280ff510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322e00d700 1 -- 172.21.15.93:0/2745194852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280fe4f0 msgr2=0x7f3228100ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322e00d700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280fe4f0 0x7f3228100ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322e00d700 1 -- 172.21.15.93:0/2745194852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3220009d00 con 0x7f3228101ef0 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322e00d700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 0x7f32280ffa50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3220009fd0 tx=0x7f322000bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f321effd700 1 -- 172.21.15.93:0/2745194852 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322000e040 con 0x7f3228101ef0 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f322810bbe0 con 0x7f3228101ef0 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f321effd700 1 -- 172.21.15.93:0/2745194852 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f3220003e10 con 0x7f3228101ef0 2023-12-14T18:25:55.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f321effd700 1 -- 172.21.15.93:0/2745194852 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f322001b750 con 0x7f3228101ef0 2023-12-14T18:25:55.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.275+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f322810ba50 con 0x7f3228101ef0 2023-12-14T18:25:55.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.276+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32281072a0 con 0x7f3228101ef0 2023-12-14T18:25:55.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.277+0000 7f321effd700 1 -- 172.21.15.93:0/2745194852 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f322001da30 con 0x7f3228101ef0 2023-12-14T18:25:55.280 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.278+0000 7f321effd700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32140a1260 0x7f32140a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.278+0000 7f322d80c700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32140a1260 0x7f32140a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.278+0000 7f321effd700 1 -- 172.21.15.93:0/2745194852 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f32200ce930 con 0x7f3228101ef0 2023-12-14T18:25:55.281 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.278+0000 7f322d80c700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32140a1260 0x7f32140a3710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3228067b10 tx=0x7f3218009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.281+0000 7f321effd700 1 -- 172.21.15.93:0/2745194852 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f3220097c40 con 0x7f3228101ef0 2023-12-14T18:25:55.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:55 smithi097 ceph-mon[43486]: pgmap v265: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 80 B/s, 0 objects/s recovering 2023-12-14T18:25:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:55 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3447375235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:55 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/573111674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:55 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1950024664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:55.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.471+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3228067160 con 0x7f3228101ef0 2023-12-14T18:25:55.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.471+0000 7f321effd700 1 -- 172.21.15.93:0/2745194852 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f322009b260 con 0x7f3228101ef0 2023-12-14T18:25:55.472 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-12-14T18:25:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32140a1260 msgr2=0x7f32140a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32140a1260 0x7f32140a3710 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3228067b10 tx=0x7f3218009260 comp rx=0 tx=0).stop 2023-12-14T18:25:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 msgr2=0x7f32280ffa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 0x7f32280ffa50 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f3220009fd0 tx=0x7f322000bce0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 shutdown_connections 2023-12-14T18:25:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f32140a1260 0x7f32140a3710 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f3228101ef0 0x7f32280ffa50 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32280fe4f0 0x7f3228100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 --2- 172.21.15.93:0/2745194852 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f3228101560 0x7f32280ff510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 >> 172.21.15.93:0/2745194852 conn(0x7f32280f9ed0 msgr2=0x7f32280fc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:55.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 shutdown_connections 2023-12-14T18:25:55.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.473+0000 7f322fa70700 1 -- 172.21.15.93:0/2745194852 wait complete. 2023-12-14T18:25:55.486 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:25:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:55 smithi160 ceph-mon[43840]: pgmap v265: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s; 80 B/s, 0 objects/s recovering 2023-12-14T18:25:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:55 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3447375235' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:55 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/573111674' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:55 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1950024664' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:25:55.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.610+0000 7f9e66845700 1 -- 172.21.15.93:0/1099258319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e600ff120 msgr2=0x7f9e600ff530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.610+0000 7f9e66845700 1 --2- 172.21.15.93:0/1099258319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e600ff120 0x7f9e600ff530 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9e50009990 tx=0x7f9e50009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.610+0000 7f9e66845700 1 -- 172.21.15.93:0/1099258319 shutdown_connections 2023-12-14T18:25:55.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.610+0000 7f9e66845700 1 --2- 172.21.15.93:0/1099258319 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e600ff120 0x7f9e600ff530 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.610+0000 7f9e66845700 1 --2- 172.21.15.93:0/1099258319 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e60100ca0 0x7f9e60107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.610+0000 7f9e66845700 1 --2- 172.21.15.93:0/1099258319 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e60100310 0x7f9e60100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.614 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.610+0000 7f9e66845700 1 -- 172.21.15.93:0/1099258319 >> 172.21.15.93:0/1099258319 conn(0x7f9e600fa740 msgr2=0x7f9e600fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.610+0000 7f9e66845700 1 -- 172.21.15.93:0/1099258319 shutdown_connections 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.610+0000 7f9e66845700 1 -- 172.21.15.93:0/1099258319 wait complete. 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e66845700 1 Processor -- start 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e66845700 1 -- start start 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e66845700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e600ff120 0x7f9e60194eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e66845700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e60100310 0x7f9e601953f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e66845700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e60100ca0 0x7f9e6019c460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e66845700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6019ca30 con 0x7f9e60100ca0 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e66845700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6019cb70 con 0x7f9e600ff120 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e66845700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e6019cd00 con 0x7f9e60100310 2023-12-14T18:25:55.615 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e64de2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e60100ca0 0x7f9e6019c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e64de2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e60100ca0 0x7f9e6019c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47026/0 (socket says 172.21.15.93:47026) 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.611+0000 7f9e64de2700 1 -- 172.21.15.93:0/3704604944 learned_addr learned my addr 172.21.15.93:0/3704604944 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e5ffff700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e600ff120 0x7f9e60194eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e5f7fe700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e60100310 0x7f9e601953f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e64de2700 1 -- 172.21.15.93:0/3704604944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e60100310 msgr2=0x7f9e601953f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e64de2700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e60100310 0x7f9e601953f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e64de2700 1 -- 172.21.15.93:0/3704604944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e600ff120 msgr2=0x7f9e60194eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e64de2700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e600ff120 0x7f9e60194eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e64de2700 1 -- 172.21.15.93:0/3704604944 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e500095f0 con 0x7f9e60100ca0 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e5f7fe700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e60100310 0x7f9e601953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:55.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e64de2700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e60100ca0 0x7f9e6019c460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800efb0 tx=0x7f9e5800c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e5d7fa700 1 -- 172.21.15.93:0/3704604944 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e58011670 con 0x7f9e60100ca0 2023-12-14T18:25:55.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e5d7fa700 1 -- 172.21.15.93:0/3704604944 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9e58011c90 con 0x7f9e60100ca0 2023-12-14T18:25:55.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e6010aae0 con 0x7f9e60100ca0 2023-12-14T18:25:55.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e5d7fa700 1 -- 172.21.15.93:0/3704604944 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e5801ac70 con 0x7f9e60100ca0 2023-12-14T18:25:55.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.612+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e6010a950 con 0x7f9e60100ca0 2023-12-14T18:25:55.617 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.613+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e60068a20 con 0x7f9e60100ca0 2023-12-14T18:25:55.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.614+0000 7f9e5d7fa700 1 -- 172.21.15.93:0/3704604944 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9e580117d0 con 0x7f9e60100ca0 2023-12-14T18:25:55.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.614+0000 7f9e5d7fa700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9e480a11e0 0x7f9e480a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.615+0000 7f9e5ffff700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9e480a11e0 0x7f9e480a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.615+0000 7f9e5d7fa700 1 -- 172.21.15.93:0/3704604944 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f9e580cf4e0 con 0x7f9e60100ca0 2023-12-14T18:25:55.618 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.615+0000 7f9e5ffff700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9e480a11e0 0x7f9e480a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000b5c0 tx=0x7f9e50005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.619+0000 7f9e5d7fa700 1 -- 172.21.15.93:0/3704604944 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9e58098760 con 0x7f9e60100ca0 2023-12-14T18:25:55.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.809+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9e60105f40 con 0x7f9e60100ca0 2023-12-14T18:25:55.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.809+0000 7f9e5d7fa700 1 -- 172.21.15.93:0/3704604944 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9e5809bd80 con 0x7f9e60100ca0 2023-12-14T18:25:55.810 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:25:55.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9e480a11e0 msgr2=0x7f9e480a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9e480a11e0 0x7f9e480a3690 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f9e5000b5c0 tx=0x7f9e50005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e60100ca0 msgr2=0x7f9e6019c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e60100ca0 0x7f9e6019c460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9e5800efb0 tx=0x7f9e5800c550 comp rx=0 tx=0).stop 2023-12-14T18:25:55.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 shutdown_connections 2023-12-14T18:25:55.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f9e480a11e0 0x7f9e480a3690 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9e60100ca0 0x7f9e6019c460 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9e600ff120 0x7f9e60194eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 --2- 172.21.15.93:0/3704604944 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9e60100310 0x7f9e601953f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 >> 172.21.15.93:0/3704604944 conn(0x7f9e600fa740 msgr2=0x7f9e600fc9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:55.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 shutdown_connections 2023-12-14T18:25:55.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.811+0000 7f9e66845700 1 -- 172.21.15.93:0/3704604944 wait complete. 2023-12-14T18:25:55.825 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:55.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.946+0000 7f35808a5700 1 -- 172.21.15.93:0/1868603486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101430 msgr2=0x7f357c101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.946+0000 7f35808a5700 1 --2- 172.21.15.93:0/1868603486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101430 0x7f357c101880 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3570009990 tx=0x7f3570009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.946+0000 7f35808a5700 1 -- 172.21.15.93:0/1868603486 shutdown_connections 2023-12-14T18:25:55.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.946+0000 7f35808a5700 1 --2- 172.21.15.93:0/1868603486 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357c101dc0 0x7f357c108680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.946+0000 7f35808a5700 1 --2- 172.21.15.93:0/1868603486 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f357c100240 0x7f357c100650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.946+0000 7f35808a5700 1 --2- 172.21.15.93:0/1868603486 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101430 0x7f357c101880 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.946+0000 7f35808a5700 1 -- 172.21.15.93:0/1868603486 >> 172.21.15.93:0/1868603486 conn(0x7f357c0fb820 msgr2=0x7f357c0fdc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:55.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.946+0000 7f35808a5700 1 -- 172.21.15.93:0/1868603486 shutdown_connections 2023-12-14T18:25:55.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.946+0000 7f35808a5700 1 -- 172.21.15.93:0/1868603486 wait complete. 2023-12-14T18:25:55.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f35808a5700 1 Processor -- start 2023-12-14T18:25:55.948 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f35808a5700 1 -- start start 2023-12-14T18:25:55.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f35808a5700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357c100240 0x7f357c195eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.949 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f35808a5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f357c101430 0x7f357c1963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f35808a5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101dc0 0x7f357c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f35808a5700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c19da30 con 0x7f357c101dc0 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f35808a5700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c19db70 con 0x7f357c100240 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f35808a5700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f357c19dd00 con 0x7f357c101430 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f357ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101dc0 0x7f357c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f357ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101dc0 0x7f357c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47050/0 (socket says 172.21.15.93:47050) 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f357ad9d700 1 -- 172.21.15.93:0/1154785800 learned_addr learned my addr 172.21.15.93:0/1154785800 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f3579d9b700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f357c101430 0x7f357c1963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f357a59c700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357c100240 0x7f357c195eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f357ad9d700 1 -- 172.21.15.93:0/1154785800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f357c101430 msgr2=0x7f357c1963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f357ad9d700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f357c101430 0x7f357c1963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f357ad9d700 1 -- 172.21.15.93:0/1154785800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357c100240 msgr2=0x7f357c195eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f357ad9d700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357c100240 0x7f357c195eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.947+0000 7f357ad9d700 1 -- 172.21.15.93:0/1154785800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35700095f0 con 0x7f357c101dc0 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.948+0000 7f357ad9d700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101dc0 0x7f357c19d460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3574007d70 tx=0x7f357400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.948+0000 7f356b7fe700 1 -- 172.21.15.93:0/1154785800 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f357400ad40 con 0x7f357c101dc0 2023-12-14T18:25:55.950 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.948+0000 7f356b7fe700 1 -- 172.21.15.93:0/1154785800 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f35740128e0 con 0x7f357c101dc0 2023-12-14T18:25:55.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.948+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f357c10bba0 con 0x7f357c101dc0 2023-12-14T18:25:55.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.948+0000 7f356b7fe700 1 -- 172.21.15.93:0/1154785800 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35740116b0 con 0x7f357c101dc0 2023-12-14T18:25:55.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.948+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f357c10ba10 con 0x7f357c101dc0 2023-12-14T18:25:55.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.949+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f357c0679f0 con 0x7f357c101dc0 2023-12-14T18:25:55.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.950+0000 7f356b7fe700 1 -- 172.21.15.93:0/1154785800 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f3574012420 con 0x7f357c101dc0 2023-12-14T18:25:55.951 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.950+0000 7f356b7fe700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f35640a11e0 0x7f35640a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:55.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.950+0000 7f357a59c700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f35640a11e0 0x7f35640a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:55.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.951+0000 7f356b7fe700 1 -- 172.21.15.93:0/1154785800 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f35740cd300 con 0x7f357c101dc0 2023-12-14T18:25:55.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.951+0000 7f357a59c700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f35640a11e0 0x7f35640a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f356c005950 tx=0x7f356c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:55.954 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:55.954+0000 7f356b7fe700 1 -- 172.21.15.93:0/1154785800 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f35740966f0 con 0x7f357c101dc0 2023-12-14T18:25:56.190 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.190+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f357c19e760 con 0x7f357c101dc0 2023-12-14T18:25:56.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.190+0000 7f356b7fe700 1 -- 172.21.15.93:0/1154785800 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10799 (secure 0 0 0) 0x7f3574023350 con 0x7f357c101dc0 2023-12-14T18:25:56.191 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:56.191 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:25:01.760987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.192+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f35640a11e0 msgr2=0x7f35640a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.192+0000 7f35808a5700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f35640a11e0 0x7f35640a3690 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f356c005950 tx=0x7f356c0058e0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.192+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101dc0 msgr2=0x7f357c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.192+0000 7f35808a5700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101dc0 0x7f357c19d460 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3574007d70 tx=0x7f357400a340 comp rx=0 tx=0).stop 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.193+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 shutdown_connections 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.193+0000 7f35808a5700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f35640a11e0 0x7f35640a3690 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.193+0000 7f35808a5700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f357c101dc0 0x7f357c19d460 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.193+0000 7f35808a5700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f357c100240 0x7f357c195eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.193+0000 7f35808a5700 1 --2- 172.21.15.93:0/1154785800 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f357c101430 0x7f357c1963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.193+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 >> 172.21.15.93:0/1154785800 conn(0x7f357c0fb820 msgr2=0x7f357c0fda40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.193+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 shutdown_connections 2023-12-14T18:25:56.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.193+0000 7f35808a5700 1 -- 172.21.15.93:0/1154785800 wait complete. 2023-12-14T18:25:56.195 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 24 2023-12-14T18:25:56.205 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:56 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3854766733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:25:56.205 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:56 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2745194852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:25:56.205 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:56 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3704604944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:25:56.206 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:25:56.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.326+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/4002788215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e80ff140 msgr2=0x7fa2e80ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.326+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/4002788215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e80ff140 0x7fa2e80ff550 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8009990 tx=0x7fa2d8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.327 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.326+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/4002788215 shutdown_connections 2023-12-14T18:25:56.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.326+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/4002788215 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e80ff140 0x7fa2e80ff550 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.326+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/4002788215 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2e8100cc0 0x7fa2e8107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.326+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/4002788215 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e8100330 0x7fa2e8100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.326+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/4002788215 >> 172.21.15.93:0/4002788215 conn(0x7fa2e80fa760 msgr2=0x7fa2e80fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:56.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.326+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/4002788215 shutdown_connections 2023-12-14T18:25:56.328 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.326+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/4002788215 wait complete. 2023-12-14T18:25:56.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.327+0000 7fa2ed5e7700 1 Processor -- start 2023-12-14T18:25:56.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.327+0000 7fa2ed5e7700 1 -- start start 2023-12-14T18:25:56.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2ed5e7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e80ff140 0x7fa2e8195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2ed5e7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2e8100330 0x7fa2e8196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2ed5e7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e8100cc0 0x7fa2e819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2ed5e7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e819da40 con 0x7fa2e8100cc0 2023-12-14T18:25:56.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2ed5e7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e819db80 con 0x7fa2e8100330 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2ed5e7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2e819dd10 con 0x7fa2e80ff140 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e8100cc0 0x7fa2e819d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e8100cc0 0x7fa2e819d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47076/0 (socket says 172.21.15.93:47076) 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e77fe700 1 -- 172.21.15.93:0/1731501179 learned_addr learned my addr 172.21.15.93:0/1731501179 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e67fc700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2e8100330 0x7fa2e8196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e6ffd700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e80ff140 0x7fa2e8195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e77fe700 1 -- 172.21.15.93:0/1731501179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e80ff140 msgr2=0x7fa2e8195ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e77fe700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e80ff140 0x7fa2e8195ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e77fe700 1 -- 172.21.15.93:0/1731501179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2e8100330 msgr2=0x7fa2e8196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e77fe700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2e8100330 0x7fa2e8196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e77fe700 1 -- 172.21.15.93:0/1731501179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2d80095f0 con 0x7fa2e8100cc0 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e67fc700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2e8100330 0x7fa2e8196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2e77fe700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e8100cc0 0x7fa2e819d470 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0007d70 tx=0x7fa2e000a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2cffff700 1 -- 172.21.15.93:0/1731501179 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e000ad40 con 0x7fa2e8100cc0 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2cffff700 1 -- 172.21.15.93:0/1731501179 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa2e00128e0 con 0x7fa2e8100cc0 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2e810bc20 con 0x7fa2e8100cc0 2023-12-14T18:25:56.331 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2cffff700 1 -- 172.21.15.93:0/1731501179 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0011780 con 0x7fa2e8100cc0 2023-12-14T18:25:56.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.328+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2e810ba90 con 0x7fa2e8100cc0 2023-12-14T18:25:56.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.329+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2e8106600 con 0x7fa2e8100cc0 2023-12-14T18:25:56.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.331+0000 7fa2cffff700 1 -- 172.21.15.93:0/1731501179 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fa2e0012a50 con 0x7fa2e8100cc0 2023-12-14T18:25:56.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.332+0000 7fa2cffff700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2d00a11e0 0x7fa2d00a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.332+0000 7fa2e6ffd700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2d00a11e0 0x7fa2d00a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.332+0000 7fa2cffff700 1 -- 172.21.15.93:0/1731501179 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa2e00cd3f0 con 0x7fa2e8100cc0 2023-12-14T18:25:56.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.332+0000 7fa2e6ffd700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2d00a11e0 0x7fa2d00a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006010 tx=0x7fa2d800b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:56.335 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.335+0000 7fa2cffff700 1 -- 172.21.15.93:0/1731501179 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa2e00967e0 con 0x7fa2e8100cc0 2023-12-14T18:25:56.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:56 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3854766733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:25:56.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:56 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2745194852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:25:56.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:56 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3704604944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:25:56.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:56 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3854766733' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:25:56.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:56 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2745194852' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:25:56.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:56 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3704604944' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:25:56.529 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.528+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa2e802d030 con 0x7fa2e8100cc0 2023-12-14T18:25:56.530 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.530+0000 7fa2cffff700 1 -- 172.21.15.93:0/1731501179 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24357 (secure 0 0 0) 0x7fa2e0023350 con 0x7fa2e8100cc0 2023-12-14T18:25:56.531 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:56.531 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":86,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:25:37.836200+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:25:56.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2d00a11e0 msgr2=0x7fa2d00a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2d00a11e0 0x7fa2d00a3690 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fa2d8006010 tx=0x7fa2d800b540 comp rx=0 tx=0).stop 2023-12-14T18:25:56.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e8100cc0 msgr2=0x7fa2e819d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e8100cc0 0x7fa2e819d470 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fa2e0007d70 tx=0x7fa2e000a340 comp rx=0 tx=0).stop 2023-12-14T18:25:56.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 shutdown_connections 2023-12-14T18:25:56.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa2d00a11e0 0x7fa2d00a3690 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa2e8100cc0 0x7fa2e819d470 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.533 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2e80ff140 0x7fa2e8195ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.534 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 --2- 172.21.15.93:0/1731501179 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2e8100330 0x7fa2e8196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.534 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 >> 172.21.15.93:0/1731501179 conn(0x7fa2e80fa760 msgr2=0x7fa2e8103740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:56.534 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 shutdown_connections 2023-12-14T18:25:56.534 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.532+0000 7fa2ed5e7700 1 -- 172.21.15.93:0/1731501179 wait complete. 2023-12-14T18:25:56.545 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:25:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.669+0000 7f8dae64c700 1 -- 172.21.15.93:0/3322587380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da80ff110 msgr2=0x7f8da80ff520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.669+0000 7f8dae64c700 1 --2- 172.21.15.93:0/3322587380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da80ff110 0x7f8da80ff520 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009990 tx=0x7f8d98009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.669+0000 7f8dae64c700 1 -- 172.21.15.93:0/3322587380 shutdown_connections 2023-12-14T18:25:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.669+0000 7f8dae64c700 1 --2- 172.21.15.93:0/3322587380 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da80ff110 0x7f8da80ff520 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.669+0000 7f8dae64c700 1 --2- 172.21.15.93:0/3322587380 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da8100c90 0x7f8da8107550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.669+0000 7f8dae64c700 1 --2- 172.21.15.93:0/3322587380 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8da8100300 0x7f8da8100750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.669+0000 7f8dae64c700 1 -- 172.21.15.93:0/3322587380 >> 172.21.15.93:0/3322587380 conn(0x7f8da80fa6f0 msgr2=0x7f8da80fcb60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.669+0000 7f8dae64c700 1 -- 172.21.15.93:0/3322587380 shutdown_connections 2023-12-14T18:25:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.669+0000 7f8dae64c700 1 -- 172.21.15.93:0/3322587380 wait complete. 2023-12-14T18:25:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8dae64c700 1 Processor -- start 2023-12-14T18:25:56.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8dae64c700 1 -- start start 2023-12-14T18:25:56.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8dae64c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da80ff110 0x7f8da8196070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8dae64c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da8100300 0x7f8da81965b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8dae64c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8da8100c90 0x7f8da819d620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8dae64c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da819dbf0 con 0x7f8da8100300 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8dae64c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da819dd30 con 0x7f8da8100c90 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8dae64c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8da819dec0 con 0x7f8da80ff110 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da8100300 0x7f8da81965b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8da77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da8100300 0x7f8da81965b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47094/0 (socket says 172.21.15.93:47094) 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8da77fe700 1 -- 172.21.15.93:0/1115049396 learned_addr learned my addr 172.21.15.93:0/1115049396 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8da7fff700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da80ff110 0x7f8da8196070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8dacbe9700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8da8100c90 0x7f8da819d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8da77fe700 1 -- 172.21.15.93:0/1115049396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da80ff110 msgr2=0x7f8da8196070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8da77fe700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da80ff110 0x7f8da8196070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8da77fe700 1 -- 172.21.15.93:0/1115049396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8da8100c90 msgr2=0x7f8da819d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8da77fe700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8da8100c90 0x7f8da819d620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.670+0000 7f8da77fe700 1 -- 172.21.15.93:0/1115049396 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d980095f0 con 0x7f8da8100300 2023-12-14T18:25:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.671+0000 7f8da77fe700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da8100300 0x7f8da81965b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00e9a0 tx=0x7f8d9c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.671+0000 7f8da57fa700 1 -- 172.21.15.93:0/1115049396 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c009790 con 0x7f8da8100300 2023-12-14T18:25:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.671+0000 7f8da57fa700 1 -- 172.21.15.93:0/1115049396 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8d9c010430 con 0x7f8da8100300 2023-12-14T18:25:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.671+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8da810bc50 con 0x7f8da8100300 2023-12-14T18:25:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.671+0000 7f8da57fa700 1 -- 172.21.15.93:0/1115049396 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d9c018680 con 0x7f8da8100300 2023-12-14T18:25:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.671+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8da810bac0 con 0x7f8da8100300 2023-12-14T18:25:56.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.672+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8da81066f0 con 0x7f8da8100300 2023-12-14T18:25:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.673+0000 7f8da57fa700 1 -- 172.21.15.93:0/1115049396 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f8d9c0105a0 con 0x7f8da8100300 2023-12-14T18:25:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.673+0000 7f8da57fa700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8d900a11e0 0x7f8d900a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.674+0000 7f8da7fff700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8d900a11e0 0x7f8d900a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.674+0000 7f8da57fa700 1 -- 172.21.15.93:0/1115049396 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8d9c014070 con 0x7f8da8100300 2023-12-14T18:25:56.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.674+0000 7f8da7fff700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8d900a11e0 0x7f8d900a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d98006010 tx=0x7f8d98005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:56.677 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.677+0000 7f8da57fa700 1 -- 172.21.15.93:0/1115049396 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f8d9c096440 con 0x7f8da8100300 2023-12-14T18:25:56.911 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.911+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8da802cfe0 con 0x7f8da8100300 2023-12-14T18:25:56.912 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.911+0000 7f8da57fa700 1 -- 172.21.15.93:0/1115049396 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10799 (secure 0 0 0) 0x7f8d9c026090 con 0x7f8da8100300 2023-12-14T18:25:56.912 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:25:56.912 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:25:01.760987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:25:56.914 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.913+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8d900a11e0 msgr2=0x7f8d900a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.913+0000 7f8dae64c700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8d900a11e0 0x7f8d900a3690 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f8d98006010 tx=0x7f8d98005960 comp rx=0 tx=0).stop 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.913+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da8100300 msgr2=0x7f8da81965b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.913+0000 7f8dae64c700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da8100300 0x7f8da81965b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f8d9c00e9a0 tx=0x7f8d9c00ecb0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.913+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 shutdown_connections 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.914+0000 7f8dae64c700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8d900a11e0 0x7f8d900a3690 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.914+0000 7f8dae64c700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8da8100c90 0x7f8da819d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.914+0000 7f8dae64c700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8da80ff110 0x7f8da8196070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.914+0000 7f8dae64c700 1 --2- 172.21.15.93:0/1115049396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8da8100300 0x7f8da81965b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.914+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 >> 172.21.15.93:0/1115049396 conn(0x7f8da80fa6f0 msgr2=0x7f8da8103830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.914+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 shutdown_connections 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:56.914+0000 7f8dae64c700 1 -- 172.21.15.93:0/1115049396 wait complete. 2023-12-14T18:25:56.915 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 24 2023-12-14T18:25:56.927 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-14T18:24:36.557875+0000', 'modified': '2023-12-14T18:25:01.760987+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24461, 'mds_1': 24442, 'mds_2': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/2285100381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 2285100381}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 2285100381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.97:6835/1146986690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 1146986690}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 1146986690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6837/233011976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 233011976}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 233011976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/3050538896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3050538896}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3050538896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.160:6839/3370815452', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 3370815452}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 3370815452}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6839/3753662808', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3753662808}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3753662808}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-14T18:25:56.927 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-12-14T18:25:56.927 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-12-14T18:25:57.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.050+0000 7fc9f0b39700 1 -- 172.21.15.93:0/2689634049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 msgr2=0x7fc9ec1007b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.050+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/2689634049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 0x7fc9ec1007b0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009990 tx=0x7fc9dc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.050+0000 7fc9f0b39700 1 -- 172.21.15.93:0/2689634049 shutdown_connections 2023-12-14T18:25:57.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.050+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/2689634049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec100cf0 0x7fc9ec107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.050+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/2689634049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9ec0ff630 0x7fc9ec0ffa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.050+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/2689634049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 0x7fc9ec1007b0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.050+0000 7fc9f0b39700 1 -- 172.21.15.93:0/2689634049 >> 172.21.15.93:0/2689634049 conn(0x7fc9ec0f9ed0 msgr2=0x7fc9ec0fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.051 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.050+0000 7fc9f0b39700 1 -- 172.21.15.93:0/2689634049 shutdown_connections 2023-12-14T18:25:57.052 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.050+0000 7fc9f0b39700 1 -- 172.21.15.93:0/2689634049 wait complete. 2023-12-14T18:25:57.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9f0b39700 1 Processor -- start 2023-12-14T18:25:57.053 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9f0b39700 1 -- start start 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9f0b39700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec0ff630 0x7fc9ec196020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9f0b39700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 0x7fc9ec196560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9f0b39700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9ec100cf0 0x7fc9ec19d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9f0b39700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec19dba0 con 0x7fc9ec100340 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9f0b39700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec19dce0 con 0x7fc9ec100cf0 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9f0b39700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9ec19de70 con 0x7fc9ec0ff630 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9e9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 0x7fc9ec196560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9e9d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 0x7fc9ec196560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47122/0 (socket says 172.21.15.93:47122) 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9e9d9b700 1 -- 172.21.15.93:0/4015585064 learned_addr learned my addr 172.21.15.93:0/4015585064 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec0ff630 0x7fc9ec196020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9ec100cf0 0x7fc9ec19d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9e9d9b700 1 -- 172.21.15.93:0/4015585064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec0ff630 msgr2=0x7fc9ec196020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec0ff630 0x7fc9ec196020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9e9d9b700 1 -- 172.21.15.93:0/4015585064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9ec100cf0 msgr2=0x7fc9ec19d5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.054 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9ec100cf0 0x7fc9ec19d5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9e9d9b700 1 -- 172.21.15.93:0/4015585064 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc0095f0 con 0x7fc9ec100340 2023-12-14T18:25:57.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.051+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9ec100cf0 0x7fc9ec19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:57.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.052+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 0x7fc9ec196560 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0095c0 tx=0x7fc9dc00f690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.052+0000 7fc9db7fe700 1 -- 172.21.15.93:0/4015585064 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc01d070 con 0x7fc9ec100340 2023-12-14T18:25:57.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.052+0000 7fc9f0b39700 1 -- 172.21.15.93:0/4015585064 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9ec10aad0 con 0x7fc9ec100340 2023-12-14T18:25:57.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.052+0000 7fc9db7fe700 1 -- 172.21.15.93:0/4015585064 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9dc004830 con 0x7fc9ec100340 2023-12-14T18:25:57.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.052+0000 7fc9db7fe700 1 -- 172.21.15.93:0/4015585064 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9dc005420 con 0x7fc9ec100340 2023-12-14T18:25:57.055 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.052+0000 7fc9f0b39700 1 -- 172.21.15.93:0/4015585064 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9ec10a940 con 0x7fc9ec100340 2023-12-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.054+0000 7fc9db7fe700 1 -- 172.21.15.93:0/4015585064 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc9dc0049a0 con 0x7fc9ec100340 2023-12-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.055+0000 7fc9db7fe700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc9d40a11e0 0x7fc9d40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.055+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc9d40a11e0 0x7fc9d40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.056 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.055+0000 7fc9db7fe700 1 -- 172.21.15.93:0/4015585064 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9dc0cdfe0 con 0x7fc9ec100340 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.055+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc9d40a11e0 0x7fc9d40a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec067940 tx=0x7fc9e000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9fccc7b80 1 Processor -- start 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9fccc7b80 1 -- start start 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 0x56096ccedcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096cceeb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096ccef530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cbd2590 con 0x56096ccef0c0 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cc41820 con 0x56096cced8c0 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cc3acb0 con 0x56096ccee210 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9ead9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096ccef530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.058 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9ead9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096ccef530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47130/0 (socket says 172.21.15.93:47130) 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9ead9d700 1 -- 172.21.15.93:0/2493393915 learned_addr learned my addr 172.21.15.93:0/2493393915 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096cceeb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 0x56096ccedcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9ead9d700 1 -- 172.21.15.93:0/2493393915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 msgr2=0x56096cceeb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.056+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096cceeb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.057+0000 7fc9ead9d700 1 -- 172.21.15.93:0/2493393915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 msgr2=0x56096ccedcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.057+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 0x56096ccedcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.057+0000 7fc9ead9d700 1 -- 172.21.15.93:0/2493393915 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56096cabc360 con 0x56096ccef0c0 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.057+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 0x56096ccedcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.057+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096ccef530 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009e60 tx=0x7fc9e40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbdcfb1ce26441de server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.057+0000 7fc9d8ff9700 1 -- 172.21.15.93:0/2493393915 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4014610 con 0x56096ccef0c0 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.057+0000 7fc9d8ff9700 1 -- 172.21.15.93:0/2493393915 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e400f050 con 0x56096ccef0c0 2023-12-14T18:25:57.059 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.057+0000 7fc9d8ff9700 1 -- 172.21.15.93:0/2493393915 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4013990 con 0x56096ccef0c0 2023-12-14T18:25:57.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2493393915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 msgr2=0x56096ccef530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.063 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096ccef530 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009e60 tx=0x7fc9e40077e0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2493393915 shutdown_connections 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096ccef530 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 0x56096ccedcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/2493393915 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096cceeb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2493393915 >> 172.21.15.93:0/2493393915 conn(0x56096ccdec50 msgr2=0x56096cce43e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2493393915 shutdown_connections 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2493393915 wait complete. 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 -- start start 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 Processor -- start 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccf7a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 0x56096ccfea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cbd6f40 con 0x56096cced8c0 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cbd6be0 con 0x56096ccef0c0 2023-12-14T18:25:57.064 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cabe650 con 0x56096ccee210 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9fccc7b80 10 client.0 _ll_register_callbacks cb 0x56096cbfa8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9ea59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9ea59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf74d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47140/0 (socket says 172.21.15.93:47140) 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.058+0000 7fc9ea59c700 1 -- 172.21.15.93:0/3178896434 learned_addr learned my addr 172.21.15.93:0/3178896434 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 0x56096ccfea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccf7a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9ea59c700 1 -- 172.21.15.93:0/3178896434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 msgr2=0x56096ccf7a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccf7a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9ea59c700 1 -- 172.21.15.93:0/3178896434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 msgr2=0x56096ccfea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 0x56096ccfea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9ea59c700 1 -- 172.21.15.93:0/3178896434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9e4009a90 con 0x56096cced8c0 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccf7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf74d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00116f0 tx=0x7fc9e00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97fec9ec50fccb51 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:25:57.065 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/3178896434 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e001f840 con 0x56096cced8c0 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/3178896434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56096ccdc260 con 0x56096cced8c0 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/3178896434 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9e0017070 con 0x56096cced8c0 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/3178896434 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e0010e80 con 0x56096cced8c0 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9fccc7b80 10 client.15225 fetch_fsmap learned FSMap version 24 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/3178896434 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56096cc01120 con 0x56096cced8c0 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/3178896434 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56096ccdc260 con 0x56096cced8c0 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.059+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/3178896434 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9e0028bd0 con 0x56096cced8c0 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.060+0000 7fc9fccc7b80 10 client.15225 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.060+0000 7fc9fccc7b80 10 client.15225 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.060+0000 7fc9fccc7b80 20 client.15225 populate_metadata read hostname 'smithi093' 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.060+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x56096cd027b0 0x56096cd01bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.060+0000 7fc9fccc7b80 4 client.15225 mds_command: new command op to 24422 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.060+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/3178896434 --> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56096cc33c90 con 0x56096cd027b0 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.060+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x56096cd027b0 0x56096cd01bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.066 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.060+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x56096cd027b0 0x56096cd01bb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=fac9bff0bb8da20c server_cookie=6ff29940808e0938 in_seq=0 out_seq=0 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.060+0000 7fc9d1ffb700 10 client.15225 ms_handle_connect on v2:172.21.15.97:6834/1146986690 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/3178896434 <== mds.2 v2:172.21.15.97:6834/1146986690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56096cc33c90 con 0x56096cd027b0 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9d1ffb700 10 client.15225 handle_command_reply: tid=0 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 client.15225 shutdown 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/3178896434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 msgr2=0x56096ccf74d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf74d0 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00116f0 tx=0x7fc9e00047c0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/3178896434 shutdown_connections 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x56096cd027b0 0x56096cd01bb0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf74d0 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9d1ffb700 0 client.15225 ms_handle_reset on v2:172.21.15.97:6834/1146986690 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 0x56096ccfea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/3178896434 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccf7a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/3178896434 >> 172.21.15.93:0/3178896434 conn(0x56096ccdec50 msgr2=0x56096cce2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/3178896434 shutdown_connections 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/3178896434 wait complete. 2023-12-14T18:25:57.067 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.061+0000 7fc9fccc7b80 20 client.15225 trim_cache size 0 max 16384 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9fccc7b80 1 Processor -- start 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9fccc7b80 1 -- start start 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf5420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccfea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 0x56096cd01bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cbf9290 con 0x56096cced8c0 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cb46c50 con 0x56096ccef0c0 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cceef40 con 0x56096ccee210 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ea59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ea59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47154/0 (socket says 172.21.15.93:47154) 2023-12-14T18:25:57.085 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ea59c700 1 -- 172.21.15.93:0/2035760647 learned_addr learned my addr 172.21.15.93:0/2035760647 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccfea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 0x56096cd01bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ea59c700 1 -- 172.21.15.93:0/2035760647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 msgr2=0x56096ccfea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccfea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ea59c700 1 -- 172.21.15.93:0/2035760647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 msgr2=0x56096cd01bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 0x56096cd01bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ea59c700 1 -- 172.21.15.93:0/2035760647 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56096ccef760 con 0x56096cced8c0 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.079+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 0x56096cd01bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf5420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00116c0 tx=0x7fc9e000be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eec4f5794e8fe8ad server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9d97fa700 1 -- 172.21.15.93:0/2035760647 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e001ea00 con 0x56096cced8c0 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9d97fa700 1 -- 172.21.15.93:0/2035760647 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e0015070 con 0x56096cced8c0 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9d97fa700 1 -- 172.21.15.93:0/2035760647 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e001fac0 con 0x56096cced8c0 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2035760647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 msgr2=0x56096ccf5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf5420 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00116c0 tx=0x7fc9e000be40 comp rx=0 tx=0).stop 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2035760647 shutdown_connections 2023-12-14T18:25:57.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096cced8c0 0x56096ccf5420 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096ccef0c0 0x56096cd01bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/2035760647 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccfea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2035760647 >> 172.21.15.93:0/2035760647 conn(0x56096ccdec50 msgr2=0x56096ccff220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2035760647 shutdown_connections 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.080+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/2035760647 wait complete. 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9fccc7b80 1 -- start start 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9fccc7b80 1 Processor -- start 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 0x56096ccf96a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccf9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9fccc7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096cd56670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096ccfec10 con 0x56096ccef0c0 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096ccfed80 con 0x56096cced8c0 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9fccc7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56096cd56bb0 con 0x56096ccee210 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9fccc7b80 10 client.0 _ll_register_callbacks cb 0x7fc9e0010ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:25:57.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9ead9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096cd56670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9ead9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096cd56670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47164/0 (socket says 172.21.15.93:47164) 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9ead9d700 1 -- 172.21.15.93:0/165772133 learned_addr learned my addr 172.21.15.93:0/165772133 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 0x56096ccf96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9e9d9b700 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccf9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9ead9d700 1 -- 172.21.15.93:0/165772133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 msgr2=0x56096ccf9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccf9be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9ead9d700 1 -- 172.21.15.93:0/165772133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 msgr2=0x56096ccf96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 0x56096ccf96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.081+0000 7fc9ead9d700 1 -- 172.21.15.93:0/165772133 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56096cc01120 con 0x56096ccef0c0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9ead9d700 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096cd56670 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009010 tx=0x7fc9e4005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60c78ac495b255d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9d1ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/165772133 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4022070 con 0x56096ccef0c0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/165772133 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc9e40098c0 con 0x56096ccef0c0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/165772133 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fc9e400f050 con 0x56096ccef0c0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/165772133 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc9e40090b0 con 0x56096ccef0c0 2023-12-14T18:25:57.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9fccc7b80 10 client.15231 fetch_fsmap learned FSMap version 24 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/165772133 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56096ca68b10 con 0x56096ccef0c0 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/165772133 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc9e400f050 con 0x56096ccef0c0 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/165772133 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fc9e4014b60 con 0x56096ccef0c0 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9fccc7b80 10 client.15231 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9fccc7b80 10 client.15231 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9fccc7b80 20 client.15231 populate_metadata read hostname 'smithi093' 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x56096cd5b880 0x56096cd581f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9fccc7b80 4 client.15231 mds_command: new command op to 24422 tid=0[{"prefix": "status"}] 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.082+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/165772133 --> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56096cc339b0 con 0x56096cd5b880 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x56096cd5b880 0x56096cd581f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9d1ffb700 10 client.15231 ms_handle_connect on v2:172.21.15.97:6834/1146986690 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9ea59c700 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x56096cd5b880 0x56096cd581f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=1c2fd0431d651c94 server_cookie=5b47b61c14ec9df8 in_seq=0 out_seq=0 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9d1ffb700 1 -- 172.21.15.93:0/165772133 <== mds.2 v2:172.21.15.97:6834/1146986690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x56096cc339b0 con 0x56096cd5b880 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9d1ffb700 10 client.15231 handle_command_reply: tid=0 2023-12-14T18:25:57.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 client.15231 shutdown 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/165772133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 msgr2=0x56096cd56670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096cd56670 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7fc9e4009010 tx=0x7fc9e4005270 comp rx=0 tx=0).stop 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/165772133 shutdown_connections 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x56096cd5b880 0x56096cd581f0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56096ccef0c0 0x56096cd56670 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9d1ffb700 0 client.15231 ms_handle_reset on v2:172.21.15.97:6834/1146986690 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56096cced8c0 0x56096ccf96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 --2- 172.21.15.93:0/165772133 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56096ccee210 0x56096ccf9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/165772133 >> 172.21.15.93:0/165772133 conn(0x56096ccdec50 msgr2=0x56096cd01bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/165772133 shutdown_connections 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 1 -- 172.21.15.93:0/165772133 wait complete. 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.083+0000 7fc9fccc7b80 20 client.15231 trim_cache size 0 max 16384 2023-12-14T18:25:57.090 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 2, 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24422, 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 56.325865821000001, 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 24, 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 86, 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 84, 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 98.511407324000004 2023-12-14T18:25:57.091 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:25:57.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 -- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc9d40a11e0 msgr2=0x7fc9d40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc9d40a11e0 0x7fc9d40a3690 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fc9ec067940 tx=0x7fc9e000b410 comp rx=0 tx=0).stop 2023-12-14T18:25:57.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 -- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 msgr2=0x7fc9ec196560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 0x7fc9ec196560 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc0095c0 tx=0x7fc9dc00f690 comp rx=0 tx=0).stop 2023-12-14T18:25:57.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 -- 172.21.15.93:0/4015585064 shutdown_connections 2023-12-14T18:25:57.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc9d40a11e0 0x7fc9d40a3690 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc9ec100cf0 0x7fc9ec19d5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc9ec0ff630 0x7fc9ec196020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 --2- 172.21.15.93:0/4015585064 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc9ec100340 0x7fc9ec196560 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 -- 172.21.15.93:0/4015585064 >> 172.21.15.93:0/4015585064 conn(0x7fc9ec0f9ed0 msgr2=0x7fc9ec0fc0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 -- 172.21.15.93:0/4015585064 shutdown_connections 2023-12-14T18:25:57.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.084+0000 7fc9f0b39700 1 -- 172.21.15.93:0/4015585064 wait complete. 2023-12-14T18:25:57.097 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 -- 172.21.15.93:0/3237104904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c105f20 msgr2=0x7f584c108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 --2- 172.21.15.93:0/3237104904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c105f20 0x7f584c108380 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f584400b0a0 tx=0x7f584400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 -- 172.21.15.93:0/3237104904 shutdown_connections 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 --2- 172.21.15.93:0/3237104904 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c105f20 0x7f584c108380 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 --2- 172.21.15.93:0/3237104904 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f584c100ce0 0x7f584c1030c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 --2- 172.21.15.93:0/3237104904 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f584c103600 0x7f584c1059e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 -- 172.21.15.93:0/3237104904 >> 172.21.15.93:0/3237104904 conn(0x7f584c0fa780 msgr2=0x7f584c0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 -- 172.21.15.93:0/3237104904 shutdown_connections 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 -- 172.21.15.93:0/3237104904 wait complete. 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 Processor -- start 2023-12-14T18:25:57.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.218+0000 7f5851300700 1 -- start start 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5851300700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f584c100ce0 0x7f584c195fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5851300700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c103600 0x7f584c196520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5851300700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f584c105f20 0x7f584c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5851300700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c19db60 con 0x7f584c103600 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5851300700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c19dca0 con 0x7f584c105f20 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5851300700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f584c19de30 con 0x7f584c100ce0 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c103600 0x7f584c196520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c103600 0x7f584c196520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47180/0 (socket says 172.21.15.93:47180) 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584a7fc700 1 -- 172.21.15.93:0/3552761645 learned_addr learned my addr 172.21.15.93:0/3552761645 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584affd700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f584c100ce0 0x7f584c195fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584b7fe700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f584c105f20 0x7f584c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584a7fc700 1 -- 172.21.15.93:0/3552761645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f584c100ce0 msgr2=0x7f584c195fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584a7fc700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f584c100ce0 0x7f584c195fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584a7fc700 1 -- 172.21.15.93:0/3552761645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f584c105f20 msgr2=0x7f584c19d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584a7fc700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f584c105f20 0x7f584c19d590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584a7fc700 1 -- 172.21.15.93:0/3552761645 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5844009d00 con 0x7f584c103600 2023-12-14T18:25:57.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f584a7fc700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c103600 0x7f584c196520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f583c00c9b0 tx=0x7f583c00ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5833fff700 1 -- 172.21.15.93:0/3552761645 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c007b00 con 0x7f584c103600 2023-12-14T18:25:57.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5833fff700 1 -- 172.21.15.93:0/3552761645 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f583c007c60 con 0x7f584c103600 2023-12-14T18:25:57.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5833fff700 1 -- 172.21.15.93:0/3552761645 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f583c018660 con 0x7f584c103600 2023-12-14T18:25:57.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.219+0000 7f5851300700 1 -- 172.21.15.93:0/3552761645 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f584c10bc20 con 0x7f584c103600 2023-12-14T18:25:57.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.220+0000 7f5851300700 1 -- 172.21.15.93:0/3552761645 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f584c10ba90 con 0x7f584c103600 2023-12-14T18:25:57.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.221+0000 7f5833fff700 1 -- 172.21.15.93:0/3552761645 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f583c00f450 con 0x7f584c103600 2023-12-14T18:25:57.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.222+0000 7f5833fff700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58340a11e0 0x7f58340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.222+0000 7f584affd700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58340a11e0 0x7f58340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.229 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.222+0000 7f5833fff700 1 -- 172.21.15.93:0/3552761645 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f583c0ccda0 con 0x7f584c103600 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.222+0000 7f584affd700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58340a11e0 0x7f58340a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5840005950 tx=0x7f584000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f585d48eb80 1 Processor -- start 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f585d48eb80 1 -- start start 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb134b0 0x55564fb138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb13e00 0x55564fb14720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb14cb0 0x55564fb15120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f585d48eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564f9fc590 con 0x55564fb134b0 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f585d48eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa6b820 con 0x55564fb13e00 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f585d48eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa64cb0 con 0x55564fb14cb0 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f584affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb134b0 0x55564fb138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f584affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb134b0 0x55564fb138c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47196/0 (socket says 172.21.15.93:47196) 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f584affd700 1 -- 172.21.15.93:0/665783802 learned_addr learned my addr 172.21.15.93:0/665783802 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.223+0000 7f584b7fe700 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb14cb0 0x55564fb15120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f584a7fc700 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb13e00 0x55564fb14720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.230 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f584affd700 1 -- 172.21.15.93:0/665783802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb14cb0 msgr2=0x55564fb15120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f584affd700 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb14cb0 0x55564fb15120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f584affd700 1 -- 172.21.15.93:0/665783802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb13e00 msgr2=0x55564fb14720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f584affd700 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb13e00 0x55564fb14720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f584affd700 1 -- 172.21.15.93:0/665783802 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55564f8e6360 con 0x55564fb134b0 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f584b7fe700 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb14cb0 0x55564fb15120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f584affd700 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb134b0 0x55564fb138c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5840010fa0 tx=0x7f5840013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2685a02f83c520f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f58317fa700 1 -- 172.21.15.93:0/665783802 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584001e620 con 0x55564fb134b0 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f58317fa700 1 -- 172.21.15.93:0/665783802 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5840015070 con 0x55564fb134b0 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.224+0000 7f58317fa700 1 -- 172.21.15.93:0/665783802 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584001ca20 con 0x55564fb134b0 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 -- 172.21.15.93:0/665783802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb134b0 msgr2=0x55564fb138c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb134b0 0x55564fb138c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f5840010fa0 tx=0x7f5840013000 comp rx=0 tx=0).stop 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 -- 172.21.15.93:0/665783802 shutdown_connections 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb134b0 0x55564fb138c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb14cb0 0x55564fb15120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 --2- 172.21.15.93:0/665783802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb13e00 0x55564fb14720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 -- 172.21.15.93:0/665783802 >> 172.21.15.93:0/665783802 conn(0x55564fb08c50 msgr2=0x55564fb0e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 -- 172.21.15.93:0/665783802 shutdown_connections 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 -- 172.21.15.93:0/665783802 wait complete. 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 -- start start 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 Processor -- start 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb1d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb1d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb245f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa00f40 con 0x55564fb14cb0 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa00be0 con 0x55564fb134b0 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564f8e8650 con 0x55564fb13e00 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f585d48eb80 10 client.0 _ll_register_callbacks cb 0x55564fa248c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f584b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb245f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f584b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb245f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47210/0 (socket says 172.21.15.93:47210) 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.225+0000 7f584b7fe700 1 -- 172.21.15.93:0/3626696754 learned_addr learned my addr 172.21.15.93:0/3626696754 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f584affd700 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb1d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f584a7fc700 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb1d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f584b7fe700 1 -- 172.21.15.93:0/3626696754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 msgr2=0x55564fb1d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f584b7fe700 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb1d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f584b7fe700 1 -- 172.21.15.93:0/3626696754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 msgr2=0x55564fb1d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f584b7fe700 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb1d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f584b7fe700 1 -- 172.21.15.93:0/3626696754 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5840010c50 con 0x55564fb14cb0 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f584a7fc700 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb1d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f584b7fe700 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb245f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f584400b0a0 tx=0x7f5844009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac94f6da1d8580e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f582eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f582eff5700 1 -- 172.21.15.93:0/3626696754 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584400e040 con 0x55564fb14cb0 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f582eff5700 1 -- 172.21.15.93:0/3626696754 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f584401c440 con 0x55564fb14cb0 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f585d48eb80 1 -- 172.21.15.93:0/3626696754 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55564fb06260 con 0x55564fb14cb0 2023-12-14T18:25:57.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f582eff5700 1 -- 172.21.15.93:0/3626696754 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f584401c640 con 0x55564fb14cb0 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f585d48eb80 10 client.15243 fetch_fsmap learned FSMap version 24 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f585d48eb80 1 -- 172.21.15.93:0/3626696754 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55564fa2b120 con 0x55564fb14cb0 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f582eff5700 1 -- 172.21.15.93:0/3626696754 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55564fb06260 con 0x55564fb14cb0 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f582eff5700 1 -- 172.21.15.93:0/3626696754 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5844019050 con 0x55564fb14cb0 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.226+0000 7f585d48eb80 10 client.15243 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.227+0000 7f585d48eb80 10 client.15243 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.227+0000 7f585d48eb80 20 client.15243 populate_metadata read hostname 'smithi093' 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.227+0000 7f585d48eb80 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x55564fb28310 0x55564fb276f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.227+0000 7f585d48eb80 4 client.15243 mds_command: new command op to 24442 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.227+0000 7f585d48eb80 1 -- 172.21.15.93:0/3626696754 --> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55564fa5dc90 con 0x55564fb28310 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.227+0000 7f584affd700 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x55564fb28310 0x55564fb276f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.227+0000 7f582eff5700 10 client.15243 ms_handle_connect on v2:172.21.15.160:6838/3370815452 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.227+0000 7f584affd700 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x55564fb28310 0x55564fb276f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=98c0498c0d3a4eb2 server_cookie=18b25d02e927c6da in_seq=0 out_seq=0 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f582eff5700 1 -- 172.21.15.93:0/3626696754 <== mds.1 v2:172.21.15.160:6838/3370815452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55564fa5dc90 con 0x55564fb28310 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f582eff5700 10 client.15243 handle_command_reply: tid=0 2023-12-14T18:25:57.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 client.15243 shutdown 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 -- 172.21.15.93:0/3626696754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 msgr2=0x55564fb245f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb245f0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f584400b0a0 tx=0x7f5844009430 comp rx=0 tx=0).stop 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 -- 172.21.15.93:0/3626696754 shutdown_connections 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb245f0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x55564fb28310 0x55564fb276f0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb1d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f582eff5700 0 client.15243 ms_handle_reset on v2:172.21.15.160:6838/3370815452 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 --2- 172.21.15.93:0/3626696754 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb1d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 -- 172.21.15.93:0/3626696754 >> 172.21.15.93:0/3626696754 conn(0x55564fb08c50 msgr2=0x55564fb0cee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 -- 172.21.15.93:0/3626696754 shutdown_connections 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 1 -- 172.21.15.93:0/3626696754 wait complete. 2023-12-14T18:25:57.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.228+0000 7f585d48eb80 20 client.15243 trim_cache size 0 max 16384 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f585d48eb80 1 Processor -- start 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f585d48eb80 1 -- start start 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb0d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb24510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1b020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f585d48eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fb0e9e0 con 0x55564fb14cb0 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f585d48eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fb0f6b0 con 0x55564fb134b0 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f585d48eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fa33a50 con 0x55564fb13e00 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1b020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47218/0 (socket says 172.21.15.93:47218) 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584b7fe700 1 -- 172.21.15.93:0/520952259 learned_addr learned my addr 172.21.15.93:0/520952259 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584a7fc700 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb24510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.252 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584affd700 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb0d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584b7fe700 1 -- 172.21.15.93:0/520952259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 msgr2=0x55564fb24510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584b7fe700 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb24510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584b7fe700 1 -- 172.21.15.93:0/520952259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 msgr2=0x55564fb0d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584b7fe700 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb0d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.246+0000 7f584b7fe700 1 -- 172.21.15.93:0/520952259 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55564fb155a0 con 0x55564fb14cb0 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f584affd700 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb0d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f584b7fe700 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1b020 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5844009a20 tx=0x7f58440095f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1f5c24b33794e69 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f5831ffb700 1 -- 172.21.15.93:0/520952259 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584400e070 con 0x55564fb14cb0 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f5831ffb700 1 -- 172.21.15.93:0/520952259 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f584400f040 con 0x55564fb14cb0 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f5831ffb700 1 -- 172.21.15.93:0/520952259 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584401c9b0 con 0x55564fb14cb0 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f585d48eb80 1 -- 172.21.15.93:0/520952259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 msgr2=0x55564fb1b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f585d48eb80 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1b020 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f5844009a20 tx=0x7f58440095f0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f585d48eb80 1 -- 172.21.15.93:0/520952259 shutdown_connections 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f585d48eb80 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1b020 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f585d48eb80 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb0d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.253 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f585d48eb80 1 --2- 172.21.15.93:0/520952259 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb24510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f585d48eb80 1 -- 172.21.15.93:0/520952259 >> 172.21.15.93:0/520952259 conn(0x55564fb08c50 msgr2=0x55564fb1be90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f585d48eb80 1 -- 172.21.15.93:0/520952259 shutdown_connections 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.247+0000 7f585d48eb80 1 -- 172.21.15.93:0/520952259 wait complete. 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f585d48eb80 1 -- start start 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f585d48eb80 1 Processor -- start 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb1f780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f585d48eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1fd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f585d48eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fb7c190 con 0x55564fb14cb0 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f585d48eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fb7c2d0 con 0x55564fb134b0 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f585d48eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55564fb7c440 con 0x55564fb13e00 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f585d48eb80 10 client.0 _ll_register_callbacks cb 0x7f5844009f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47226/0 (socket says 172.21.15.93:47226) 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584b7fe700 1 -- 172.21.15.93:0/1475441977 learned_addr learned my addr 172.21.15.93:0/1475441977 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.254 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584affd700 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb1f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584a7fc700 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584b7fe700 1 -- 172.21.15.93:0/1475441977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 msgr2=0x55564fb24600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584b7fe700 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb24600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584b7fe700 1 -- 172.21.15.93:0/1475441977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 msgr2=0x55564fb1f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584b7fe700 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb1f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584b7fe700 1 -- 172.21.15.93:0/1475441977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55564fa2b120 con 0x55564fb14cb0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.248+0000 7f584a7fc700 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb24600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f584b7fe700 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1fd60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5844007a10 tx=0x7f584401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68485de82cfa9046 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f582eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f582eff5700 1 -- 172.21.15.93:0/1475441977 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f584400e070 con 0x55564fb14cb0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f582eff5700 1 -- 172.21.15.93:0/1475441977 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f584401cb30 con 0x55564fb14cb0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f585d48eb80 1 -- 172.21.15.93:0/1475441977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5840015070 con 0x55564fb14cb0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f582eff5700 1 -- 172.21.15.93:0/1475441977 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f584401ccf0 con 0x55564fb14cb0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f585d48eb80 10 client.15249 fetch_fsmap learned FSMap version 24 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f585d48eb80 1 -- 172.21.15.93:0/1475441977 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55564f892b10 con 0x55564fb14cb0 2023-12-14T18:25:57.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f582eff5700 1 -- 172.21.15.93:0/1475441977 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5840015070 con 0x55564fb14cb0 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f582eff5700 1 -- 172.21.15.93:0/1475441977 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f584400b4d0 con 0x55564fb14cb0 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f585d48eb80 10 client.15249 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f585d48eb80 10 client.15249 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f585d48eb80 20 client.15249 populate_metadata read hostname 'smithi093' 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f585d48eb80 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x55564fb80c00 0x55564fb7d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f585d48eb80 4 client.15249 mds_command: new command op to 24442 tid=0[{"prefix": "status"}] 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.249+0000 7f585d48eb80 1 -- 172.21.15.93:0/1475441977 --> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55564fa5d9b0 con 0x55564fb80c00 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f584affd700 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x55564fb80c00 0x55564fb7d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f584affd700 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x55564fb80c00 0x55564fb7d5c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=21ace94c375df9cf server_cookie=a5d8e80f6f75ba4a in_seq=0 out_seq=0 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f582eff5700 10 client.15249 ms_handle_connect on v2:172.21.15.160:6838/3370815452 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f582eff5700 1 -- 172.21.15.93:0/1475441977 <== mds.1 v2:172.21.15.160:6838/3370815452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+332 (crc 0 0 0) 0x55564fa5d9b0 con 0x55564fb80c00 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f582eff5700 10 client.15249 handle_command_reply: tid=0 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 client.15249 shutdown 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 -- 172.21.15.93:0/1475441977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 msgr2=0x55564fb1fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1fd60 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f5844007a10 tx=0x7f584401c8c0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.256 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 -- 172.21.15.93:0/1475441977 shutdown_connections 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55564fb14cb0 0x55564fb1fd60 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x55564fb80c00 0x55564fb7d5c0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55564fb134b0 0x55564fb1f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f582eff5700 0 client.15249 ms_handle_reset on v2:172.21.15.160:6838/3370815452 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 --2- 172.21.15.93:0/1475441977 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55564fb13e00 0x55564fb24600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 -- 172.21.15.93:0/1475441977 >> 172.21.15.93:0/1475441977 conn(0x55564fb08c50 msgr2=0x55564fb1b020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 -- 172.21.15.93:0/1475441977 shutdown_connections 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 1 -- 172.21.15.93:0/1475441977 wait complete. 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.250+0000 7f585d48eb80 20 client.15249 trim_cache size 0 max 16384 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:25:57.257 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 1, 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24442, 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 57.221260913000002, 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 24, 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 86, 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 84, 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 91.391113881999999 2023-12-14T18:25:57.258 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 -- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58340a11e0 msgr2=0x7f58340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58340a11e0 0x7f58340a3690 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f5840005950 tx=0x7f584000b410 comp rx=0 tx=0).stop 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 -- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c103600 msgr2=0x7f584c196520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c103600 0x7f584c196520 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f583c00c9b0 tx=0x7f583c00ccc0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 -- 172.21.15.93:0/3552761645 shutdown_connections 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f58340a11e0 0x7f58340a3690 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f584c105f20 0x7f584c19d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f584c100ce0 0x7f584c195fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 --2- 172.21.15.93:0/3552761645 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f584c103600 0x7f584c196520 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 -- 172.21.15.93:0/3552761645 >> 172.21.15.93:0/3552761645 conn(0x7f584c0fa780 msgr2=0x7f584c101940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 -- 172.21.15.93:0/3552761645 shutdown_connections 2023-12-14T18:25:57.259 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.251+0000 7f5851300700 1 -- 172.21.15.93:0/3552761645 wait complete. 2023-12-14T18:25:57.263 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:57 smithi093 ceph-mon[42070]: pgmap v266: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:25:57.263 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:57 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1154785800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:57.263 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:57 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1731501179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:57.263 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:57 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1115049396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:57.264 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-14T18:25:57.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:57 smithi097 ceph-mon[43486]: pgmap v266: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:25:57.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:57 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1154785800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:57.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:57 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1731501179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:57.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:57 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1115049396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:57.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.385+0000 7f5c7359e700 1 -- 172.21.15.93:0/3652269066 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c103560 msgr2=0x7f5c6c105940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.385+0000 7f5c7359e700 1 --2- 172.21.15.93:0/3652269066 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c103560 0x7f5c6c105940 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f5c60009990 tx=0x7f5c60009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.385+0000 7f5c7359e700 1 -- 172.21.15.93:0/3652269066 shutdown_connections 2023-12-14T18:25:57.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.385+0000 7f5c7359e700 1 --2- 172.21.15.93:0/3652269066 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c6c105e80 0x7f5c6c108280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.385+0000 7f5c7359e700 1 --2- 172.21.15.93:0/3652269066 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c6c100c40 0x7f5c6c103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.386 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.385+0000 7f5c7359e700 1 --2- 172.21.15.93:0/3652269066 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c103560 0x7f5c6c105940 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.385+0000 7f5c7359e700 1 -- 172.21.15.93:0/3652269066 >> 172.21.15.93:0/3652269066 conn(0x7f5c6c0fa680 msgr2=0x7f5c6c0fcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.385+0000 7f5c7359e700 1 -- 172.21.15.93:0/3652269066 shutdown_connections 2023-12-14T18:25:57.387 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.385+0000 7f5c7359e700 1 -- 172.21.15.93:0/3652269066 wait complete. 2023-12-14T18:25:57.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7359e700 1 Processor -- start 2023-12-14T18:25:57.388 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7359e700 1 -- start start 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7359e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c100c40 0x7f5c6c195ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7359e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c6c103560 0x7f5c6c196400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7359e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c6c105e80 0x7f5c6c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7359e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c19da40 con 0x7f5c6c100c40 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7359e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c19db80 con 0x7f5c6c103560 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7359e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c6c19dd10 con 0x7f5c6c105e80 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c100c40 0x7f5c6c195ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c100c40 0x7f5c6c195ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47246/0 (socket says 172.21.15.93:47246) 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7259c700 1 -- 172.21.15.93:0/257702794 learned_addr learned my addr 172.21.15.93:0/257702794 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c6c103560 0x7f5c6c196400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c72d9d700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c6c105e80 0x7f5c6c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7259c700 1 -- 172.21.15.93:0/257702794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c6c105e80 msgr2=0x7f5c6c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7259c700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c6c105e80 0x7f5c6c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7259c700 1 -- 172.21.15.93:0/257702794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c6c103560 msgr2=0x7f5c6c196400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.389 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7259c700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c6c103560 0x7f5c6c196400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c7259c700 1 -- 172.21.15.93:0/257702794 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c600095f0 con 0x7f5c6c100c40 2023-12-14T18:25:57.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.386+0000 7f5c72d9d700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c6c105e80 0x7f5c6c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:57.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.387+0000 7f5c7259c700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c100c40 0x7f5c6c195ec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400d980 tx=0x7f5c6400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.387+0000 7f5c5f7fe700 1 -- 172.21.15.93:0/257702794 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64004a30 con 0x7f5c6c100c40 2023-12-14T18:25:57.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.387+0000 7f5c5f7fe700 1 -- 172.21.15.93:0/257702794 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c640054e0 con 0x7f5c6c100c40 2023-12-14T18:25:57.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.387+0000 7f5c7359e700 1 -- 172.21.15.93:0/257702794 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c6c10bb70 con 0x7f5c6c100c40 2023-12-14T18:25:57.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.387+0000 7f5c5f7fe700 1 -- 172.21.15.93:0/257702794 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c64009c90 con 0x7f5c6c100c40 2023-12-14T18:25:57.390 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.387+0000 7f5c7359e700 1 -- 172.21.15.93:0/257702794 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c10b9e0 con 0x7f5c6c100c40 2023-12-14T18:25:57.395 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.389+0000 7f5c5f7fe700 1 -- 172.21.15.93:0/257702794 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f5c64004c20 con 0x7f5c6c100c40 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.389+0000 7f5c5f7fe700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5c580a11e0 0x7f5c580a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.389+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5c580a11e0 0x7f5c580a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.389+0000 7f5c5f7fe700 1 -- 172.21.15.93:0/257702794 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c640cccc0 con 0x7f5c6c100c40 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.390+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5c580a11e0 0x7f5c580a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000b5c0 tx=0x7f5c60005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.390+0000 7f5c8098eb80 1 Processor -- start 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.390+0000 7f5c8098eb80 1 -- start start 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 0x558f06d8ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8cfc0 0x558f06d8d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d8e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06c75590 con 0x558f06d8cfc0 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06ce4820 con 0x558f06d8c670 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06cddcb0 con 0x558f06d8de70 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8cfc0 0x558f06d8d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.396 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c71d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8cfc0 0x558f06d8d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47248/0 (socket says 172.21.15.93:47248) 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c71d9b700 1 -- 172.21.15.93:0/61367055 learned_addr learned my addr 172.21.15.93:0/61367055 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c7259c700 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 0x558f06d8ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c72d9d700 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d8e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c71d9b700 1 -- 172.21.15.93:0/61367055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 msgr2=0x558f06d8e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d8e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c71d9b700 1 -- 172.21.15.93:0/61367055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 msgr2=0x558f06d8ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 0x558f06d8ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c71d9b700 1 -- 172.21.15.93:0/61367055 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f06b5f360 con 0x558f06d8cfc0 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c7259c700 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 0x558f06d8ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8cfc0 0x558f06d8d8e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000b8d0 tx=0x7f5c60020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3393ba2f8136154 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c5cff9700 1 -- 172.21.15.93:0/61367055 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6002a070 con 0x558f06d8cfc0 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c5cff9700 1 -- 172.21.15.93:0/61367055 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c6001e040 con 0x558f06d8cfc0 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.391+0000 7f5c5cff9700 1 -- 172.21.15.93:0/61367055 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c60023940 con 0x558f06d8cfc0 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.392+0000 7f5c8098eb80 1 -- 172.21.15.93:0/61367055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8cfc0 msgr2=0x558f06d8d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.397 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.392+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8cfc0 0x558f06d8d8e0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000b8d0 tx=0x7f5c60020000 comp rx=0 tx=0).stop 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.392+0000 7f5c8098eb80 1 -- 172.21.15.93:0/61367055 shutdown_connections 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.392+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d8e2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.392+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 0x558f06d8ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.392+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/61367055 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8cfc0 0x558f06d8d8e0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.392+0000 7f5c8098eb80 1 -- 172.21.15.93:0/61367055 >> 172.21.15.93:0/61367055 conn(0x558f06d81db0 msgr2=0x558f06d87590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.392+0000 7f5c8098eb80 1 -- 172.21.15.93:0/61367055 shutdown_connections 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.392+0000 7f5c8098eb80 1 -- 172.21.15.93:0/61367055 wait complete. 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c8098eb80 1 -- start start 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c8098eb80 1 Processor -- start 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 0x558f06d961d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8cfc0 0x558f06d96710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8de70 0x558f06d9d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06c79f40 con 0x558f06d8de70 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06c79be0 con 0x558f06d8c670 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06b61650 con 0x558f06d8cfc0 2023-12-14T18:25:57.398 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c8098eb80 10 client.0 _ll_register_callbacks cb 0x558f06c9d8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c72d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8de70 0x558f06d9d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c72d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8de70 0x558f06d9d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47262/0 (socket says 172.21.15.93:47262) 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c72d9d700 1 -- 172.21.15.93:0/3885618481 learned_addr learned my addr 172.21.15.93:0/3885618481 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c7259c700 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 0x558f06d961d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8cfc0 0x558f06d96710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c72d9d700 1 -- 172.21.15.93:0/3885618481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8cfc0 msgr2=0x558f06d96710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c72d9d700 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8cfc0 0x558f06d96710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c72d9d700 1 -- 172.21.15.93:0/3885618481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 msgr2=0x558f06d961d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c72d9d700 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 0x558f06d961d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c72d9d700 1 -- 172.21.15.93:0/3885618481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c108e70 con 0x558f06d8de70 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c72d9d700 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8de70 0x558f06d9d750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007b70 tx=0x7f5c68007e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae39267d9b3a18ad server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.393+0000 7f5c55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c55ffb700 1 -- 172.21.15.93:0/3885618481 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c68003a50 con 0x558f06d8de70 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c55ffb700 1 -- 172.21.15.93:0/3885618481 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c68003bb0 con 0x558f06d8de70 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c8098eb80 1 -- 172.21.15.93:0/3885618481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558f06d7f3c0 con 0x558f06d8de70 2023-12-14T18:25:57.399 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c55ffb700 1 -- 172.21.15.93:0/3885618481 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c68003d70 con 0x558f06d8de70 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c8098eb80 10 client.15261 fetch_fsmap learned FSMap version 24 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c8098eb80 1 -- 172.21.15.93:0/3885618481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558f06ca4120 con 0x558f06d8de70 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c55ffb700 1 -- 172.21.15.93:0/3885618481 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558f06d7f3c0 con 0x558f06d8de70 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c55ffb700 1 -- 172.21.15.93:0/3885618481 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c6801c060 con 0x558f06d8de70 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c8098eb80 10 client.15261 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c8098eb80 10 client.15261 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c8098eb80 20 client.15261 populate_metadata read hostname 'smithi093' 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558f06da1550 0x558f06da0900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c8098eb80 4 client.15261 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c8098eb80 1 -- 172.21.15.93:0/3885618481 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558f06cd6c90 con 0x558f06da1550 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.394+0000 7f5c7259c700 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558f06da1550 0x558f06da0900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.395+0000 7f5c55ffb700 10 client.15261 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:25:57.400 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.395+0000 7f5c7259c700 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558f06da1550 0x558f06da0900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43085817257e8701 server_cookie=e623e021e213ac62 in_seq=0 out_seq=0 2023-12-14T18:25:57.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.395+0000 7f5c55ffb700 1 -- 172.21.15.93:0/3885618481 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558f06cd6c90 con 0x558f06da1550 2023-12-14T18:25:57.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.395+0000 7f5c55ffb700 10 client.15261 handle_command_reply: tid=0 2023-12-14T18:25:57.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.395+0000 7f5c8098eb80 1 client.15261 shutdown 2023-12-14T18:25:57.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 -- 172.21.15.93:0/3885618481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8de70 msgr2=0x558f06d9d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.401 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8de70 0x558f06d9d750 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f5c68007b70 tx=0x7f5c68007e80 comp rx=0 tx=0).stop 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 -- 172.21.15.93:0/3885618481 shutdown_connections 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558f06da1550 0x558f06da0900 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8de70 0x558f06d9d750 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8c670 0x558f06d961d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/3885618481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8cfc0 0x558f06d96710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c55ffb700 0 client.15261 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 -- 172.21.15.93:0/3885618481 >> 172.21.15.93:0/3885618481 conn(0x558f06d81db0 msgr2=0x558f06d86b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 -- 172.21.15.93:0/3885618481 shutdown_connections 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 1 -- 172.21.15.93:0/3885618481 wait complete. 2023-12-14T18:25:57.402 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.396+0000 7f5c8098eb80 20 client.15261 trim_cache size 0 max 16384 2023-12-14T18:25:57.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c8098eb80 1 Processor -- start 2023-12-14T18:25:57.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c8098eb80 1 -- start start 2023-12-14T18:25:57.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d7ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 0x558f06d9d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d86070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06be9c50 con 0x558f06d8c670 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06cd9da0 con 0x558f06d8cfc0 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06d83ed0 con 0x558f06d8de70 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d7ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d7ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47272/0 (socket says 172.21.15.93:47272) 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c7259c700 1 -- 172.21.15.93:0/516597188 learned_addr learned my addr 172.21.15.93:0/516597188 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.413+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 0x558f06d9d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c72d9d700 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d86070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c7259c700 1 -- 172.21.15.93:0/516597188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 msgr2=0x558f06d86070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c7259c700 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d86070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c7259c700 1 -- 172.21.15.93:0/516597188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 msgr2=0x558f06d9d670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c7259c700 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 0x558f06d9d670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c7259c700 1 -- 172.21.15.93:0/516597188 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558f06ca3e70 con 0x558f06d8c670 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c72d9d700 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d86070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:25:57.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c7259c700 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d7ef00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c640ca390 tx=0x7f5c6401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=188e8a5d8a02325e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c5d7fa700 1 -- 172.21.15.93:0/516597188 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640b6070 con 0x558f06d8c670 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c5d7fa700 1 -- 172.21.15.93:0/516597188 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c640b1070 con 0x558f06d8c670 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c5d7fa700 1 -- 172.21.15.93:0/516597188 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c6409d040 con 0x558f06d8c670 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c8098eb80 1 -- 172.21.15.93:0/516597188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 msgr2=0x558f06d7ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d7ef00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f5c640ca390 tx=0x7f5c6401d040 comp rx=0 tx=0).stop 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c8098eb80 1 -- 172.21.15.93:0/516597188 shutdown_connections 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d7ef00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d86070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/516597188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 0x558f06d9d670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c8098eb80 1 -- 172.21.15.93:0/516597188 >> 172.21.15.93:0/516597188 conn(0x558f06d81db0 msgr2=0x558f06daa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c8098eb80 1 -- 172.21.15.93:0/516597188 shutdown_connections 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.414+0000 7f5c8098eb80 1 -- 172.21.15.93:0/516597188 wait complete. 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c8098eb80 1 -- start start 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c8098eb80 1 Processor -- start 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d9a6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 0x558f06d9cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c8098eb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d9d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06d9d860 con 0x558f06d8c670 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06d9d9d0 con 0x558f06d8cfc0 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c8098eb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558f06df53d0 con 0x558f06d8de70 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c8098eb80 10 client.0 _ll_register_callbacks cb 0x7f5c68009da0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d9a6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c7259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d9a6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47274/0 (socket says 172.21.15.93:47274) 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c7259c700 1 -- 172.21.15.93:0/226159334 learned_addr learned my addr 172.21.15.93:0/226159334 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 0x558f06d9cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c7259c700 1 -- 172.21.15.93:0/226159334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 msgr2=0x558f06d9d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c7259c700 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d9d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c7259c700 1 -- 172.21.15.93:0/226159334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 msgr2=0x558f06d9cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c7259c700 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 0x558f06d9cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c7259c700 1 -- 172.21.15.93:0/226159334 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c6c10b9e0 con 0x558f06d8c670 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c7259c700 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d9a6a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5c64099910 tx=0x7f5c64097f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0c4d22fd0a95e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:57.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.415+0000 7f5c55ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c55ffb700 1 -- 172.21.15.93:0/226159334 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c640b6070 con 0x558f06d8c670 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c55ffb700 1 -- 172.21.15.93:0/226159334 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5c6409dc90 con 0x558f06d8c670 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c8098eb80 1 -- 172.21.15.93:0/226159334 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5c6001e040 con 0x558f06d8c670 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c55ffb700 1 -- 172.21.15.93:0/226159334 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5c6409d040 con 0x558f06d8c670 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c8098eb80 10 client.15267 fetch_fsmap learned FSMap version 24 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c8098eb80 1 -- 172.21.15.93:0/226159334 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558f06c8a950 con 0x558f06d8c670 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c55ffb700 1 -- 172.21.15.93:0/226159334 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5c6407c080 con 0x558f06d8c670 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c55ffb700 1 -- 172.21.15.93:0/226159334 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5c640b2170 con 0x558f06d8c670 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c8098eb80 10 client.15267 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c8098eb80 10 client.15267 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c8098eb80 20 client.15267 populate_metadata read hostname 'smithi093' 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558f06df9c60 0x558f06df6570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c8098eb80 4 client.15267 mds_command: new command op to 24461 tid=0[{"prefix": "status"}] 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c8098eb80 1 -- 172.21.15.93:0/226159334 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558f06cd69b0 con 0x558f06df9c60 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.416+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558f06df9c60 0x558f06df6570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:57.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c55ffb700 10 client.15267 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c71d9b700 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558f06df9c60 0x558f06df6570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90bd4339301df97c server_cookie=c187278a1a5062e4 in_seq=0 out_seq=0 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c55ffb700 1 -- 172.21.15.93:0/226159334 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x558f06cd69b0 con 0x558f06df9c60 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c55ffb700 10 client.15267 handle_command_reply: tid=0 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 client.15267 shutdown 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 -- 172.21.15.93:0/226159334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 msgr2=0x558f06d9a6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d9a6a0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f5c64099910 tx=0x7f5c64097f50 comp rx=0 tx=0).stop 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 -- 172.21.15.93:0/226159334 shutdown_connections 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558f06df9c60 0x558f06df6570 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558f06d8c670 0x558f06d9a6a0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c55ffb700 0 client.15267 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558f06d8de70 0x558f06d9d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 --2- 172.21.15.93:0/226159334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558f06d8cfc0 0x558f06d9cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 -- 172.21.15.93:0/226159334 >> 172.21.15.93:0/226159334 conn(0x558f06d81db0 msgr2=0x558f06d86070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 -- 172.21.15.93:0/226159334 shutdown_connections 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 1 -- 172.21.15.93:0/226159334 wait complete. 2023-12-14T18:25:57.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.417+0000 7f5c8098eb80 20 client.15267 trim_cache size 0 max 16384 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 0, 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24461, 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 80.856452641000004, 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 24, 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 86, 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 84, 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 86.328857649 2023-12-14T18:25:57.424 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:25:57.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 -- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5c580a11e0 msgr2=0x7f5c580a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5c580a11e0 0x7f5c580a3690 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000b5c0 tx=0x7f5c60005fb0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 -- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c100c40 msgr2=0x7f5c6c195ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:57.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c100c40 0x7f5c6c195ec0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400d980 tx=0x7f5c6400dc90 comp rx=0 tx=0).stop 2023-12-14T18:25:57.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 -- 172.21.15.93:0/257702794 shutdown_connections 2023-12-14T18:25:57.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5c580a11e0 0x7f5c580a3690 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5c6c100c40 0x7f5c6c195ec0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5c6c105e80 0x7f5c6c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 --2- 172.21.15.93:0/257702794 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5c6c103560 0x7f5c6c196400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:57.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 -- 172.21.15.93:0/257702794 >> 172.21.15.93:0/257702794 conn(0x7f5c6c0fa680 msgr2=0x7f5c6c1018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:57.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 -- 172.21.15.93:0/257702794 shutdown_connections 2023-12-14T18:25:57.426 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:57.418+0000 7f5c7359e700 1 -- 172.21.15.93:0/257702794 wait complete. 2023-12-14T18:25:57.430 INFO:tasks.cephadm:Setup complete, yielding 2023-12-14T18:25:57.431 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-14T18:25:57.439 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:25:57.440 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch status' 2023-12-14T18:25:57.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:57 smithi160 ceph-mon[43840]: pgmap v266: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:25:57.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:57 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1154785800' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:57.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:57 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1731501179' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:25:57.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:57 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1115049396' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:25:57.748 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:25:58.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.706+0000 7f9516c16700 1 -- 172.21.15.93:0/3123924018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9510100620 msgr2=0x7f9510100a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.710 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.706+0000 7f9516c16700 1 --2- 172.21.15.93:0/3123924018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9510100620 0x7f9510100a70 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009990 tx=0x7f94fc009ca0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.707+0000 7f9516c16700 1 -- 172.21.15.93:0/3123924018 shutdown_connections 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.707+0000 7f9516c16700 1 --2- 172.21.15.93:0/3123924018 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9510100fb0 0x7f9510107870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.707+0000 7f9516c16700 1 --2- 172.21.15.93:0/3123924018 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f95100ff420 0x7f95100ff830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.707+0000 7f9516c16700 1 --2- 172.21.15.93:0/3123924018 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9510100620 0x7f9510100a70 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.707+0000 7f9516c16700 1 -- 172.21.15.93:0/3123924018 >> 172.21.15.93:0/3123924018 conn(0x7f95100faa50 msgr2=0x7f95100fcea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.707+0000 7f9516c16700 1 -- 172.21.15.93:0/3123924018 shutdown_connections 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.707+0000 7f9516c16700 1 -- 172.21.15.93:0/3123924018 wait complete. 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.707+0000 7f9516c16700 1 Processor -- start 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.707+0000 7f9516c16700 1 -- start start 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f9516c16700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95100ff420 0x7f9510196220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f9516c16700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9510100620 0x7f9510196760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:58.711 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f9516c16700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9510100fb0 0x7f951019d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f9516c16700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951019ddb0 con 0x7f95100ff420 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f9516c16700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951019def0 con 0x7f9510100620 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f9516c16700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f951019e080 con 0x7f9510100fb0 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95149b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95100ff420 0x7f9510196220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95149b2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95100ff420 0x7f9510196220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47280/0 (socket says 172.21.15.93:47280) 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95149b2700 1 -- 172.21.15.93:0/2386437359 learned_addr learned my addr 172.21.15.93:0/2386437359 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f950ffff700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9510100620 0x7f9510196760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95151b3700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9510100fb0 0x7f951019d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95149b2700 1 -- 172.21.15.93:0/2386437359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9510100fb0 msgr2=0x7f951019d7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95149b2700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9510100fb0 0x7f951019d7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.712 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95149b2700 1 -- 172.21.15.93:0/2386437359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9510100620 msgr2=0x7f9510196760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95149b2700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9510100620 0x7f9510196760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95149b2700 1 -- 172.21.15.93:0/2386437359 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94fc0095f0 con 0x7f95100ff420 2023-12-14T18:25:58.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f95149b2700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95100ff420 0x7f9510196220 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f950400d9b0 tx=0x7f950400dd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:58.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f950dffb700 1 -- 172.21.15.93:0/2386437359 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950400be80 con 0x7f95100ff420 2023-12-14T18:25:58.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f950dffb700 1 -- 172.21.15.93:0/2386437359 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f95040044e0 con 0x7f95100ff420 2023-12-14T18:25:58.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f951010ad80 con 0x7f95100ff420 2023-12-14T18:25:58.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.708+0000 7f950dffb700 1 -- 172.21.15.93:0/2386437359 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9504003b40 con 0x7f95100ff420 2023-12-14T18:25:58.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.709+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f951010abf0 con 0x7f95100ff420 2023-12-14T18:25:58.713 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.709+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9510067b00 con 0x7f95100ff420 2023-12-14T18:25:58.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.710+0000 7f950dffb700 1 -- 172.21.15.93:0/2386437359 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f9504009940 con 0x7f95100ff420 2023-12-14T18:25:58.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.711+0000 7f950dffb700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95000a1260 0x7f95000a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:25:58.714 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.711+0000 7f950ffff700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95000a1260 0x7f95000a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:25:58.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.711+0000 7f950dffb700 1 -- 172.21.15.93:0/2386437359 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f95040ccaf0 con 0x7f95100ff420 2023-12-14T18:25:58.715 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.711+0000 7f950ffff700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95000a1260 0x7f95000a3710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94fc006010 tx=0x7f94fc00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:25:58.716 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.715+0000 7f950dffb700 1 -- 172.21.15.93:0/2386437359 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9504095dd0 con 0x7f95100ff420 2023-12-14T18:25:58.923 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.923+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) v1 -- 0x7f951019ec30 con 0x7f95000a1260 2023-12-14T18:25:58.924 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.924+0000 7f950dffb700 1 -- 172.21.15.93:0/2386437359 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f951019ec30 con 0x7f95000a1260 2023-12-14T18:25:58.925 INFO:teuthology.orchestra.run.smithi093.stdout:Backend: cephadm 2023-12-14T18:25:58.925 INFO:teuthology.orchestra.run.smithi093.stdout:Available: Yes 2023-12-14T18:25:58.925 INFO:teuthology.orchestra.run.smithi093.stdout:Paused: No 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95000a1260 msgr2=0x7f95000a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95000a1260 0x7f95000a3710 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94fc006010 tx=0x7f94fc00b540 comp rx=0 tx=0).stop 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95100ff420 msgr2=0x7f9510196220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95100ff420 0x7f9510196220 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f950400d9b0 tx=0x7f950400dd70 comp rx=0 tx=0).stop 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 shutdown_connections 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f95000a1260 0x7f95000a3710 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f95100ff420 0x7f9510196220 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9510100fb0 0x7f951019d7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 --2- 172.21.15.93:0/2386437359 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9510100620 0x7f9510196760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:25:58.927 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 >> 172.21.15.93:0/2386437359 conn(0x7f95100faa50 msgr2=0x7f95100690c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:25:58.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 shutdown_connections 2023-12-14T18:25:58.928 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:25:58.926+0000 7f9516c16700 1 -- 172.21.15.93:0/2386437359 wait complete. 2023-12-14T18:25:59.233 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:25:59 smithi093 ceph-mon[42070]: pgmap v267: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:25:59.303 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch ps' 2023-12-14T18:25:59.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:25:59 smithi097 ceph-mon[43486]: pgmap v267: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:25:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:25:59 smithi160 ceph-mon[43840]: pgmap v267: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:25:59.615 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:00.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:00 smithi093 ceph-mon[42070]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:00.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:00 smithi097 ceph-mon[43486]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:00.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:00 smithi160 ceph-mon[43840]: from='client.15273 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:00.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.699+0000 7fe8d0d93700 1 -- 172.21.15.93:0/3953410472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc100d70 msgr2=0x7fe8cc107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.699+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/3953410472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc100d70 0x7fe8cc107630 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fe8c4009990 tx=0x7fe8c4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.699+0000 7fe8d0d93700 1 -- 172.21.15.93:0/3953410472 shutdown_connections 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.699+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/3953410472 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc100d70 0x7fe8cc107630 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.699+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/3953410472 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8cc0ff1e0 0x7fe8cc0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.699+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/3953410472 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8cc1003e0 0x7fe8cc100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.699+0000 7fe8d0d93700 1 -- 172.21.15.93:0/3953410472 >> 172.21.15.93:0/3953410472 conn(0x7fe8cc0fa7f0 msgr2=0x7fe8cc0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.700+0000 7fe8d0d93700 1 -- 172.21.15.93:0/3953410472 shutdown_connections 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.700+0000 7fe8d0d93700 1 -- 172.21.15.93:0/3953410472 wait complete. 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.700+0000 7fe8d0d93700 1 Processor -- start 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.700+0000 7fe8d0d93700 1 -- start start 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.700+0000 7fe8d0d93700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc0ff1e0 0x7fe8cc069130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.700+0000 7fe8d0d93700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8cc1003e0 0x7fe8cc19ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.700+0000 7fe8d0d93700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8cc100d70 0x7fe8cc19d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.700+0000 7fe8d0d93700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc19db00 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8d0d93700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc19dc70 con 0x7fe8cc1003e0 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8d0d93700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8cc19de30 con 0x7fe8cc100d70 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8ca59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc0ff1e0 0x7fe8cc069130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8ca59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc0ff1e0 0x7fe8cc069130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47312/0 (socket says 172.21.15.93:47312) 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8ca59c700 1 -- 172.21.15.93:0/376971710 learned_addr learned my addr 172.21.15.93:0/376971710 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8c9d9b700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8cc1003e0 0x7fe8cc19ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8cad9d700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8cc100d70 0x7fe8cc19d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8ca59c700 1 -- 172.21.15.93:0/376971710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8cc100d70 msgr2=0x7fe8cc19d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8ca59c700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8cc100d70 0x7fe8cc19d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8ca59c700 1 -- 172.21.15.93:0/376971710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8cc1003e0 msgr2=0x7fe8cc19ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8ca59c700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8cc1003e0 0x7fe8cc19ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8ca59c700 1 -- 172.21.15.93:0/376971710 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8c40095f0 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8cad9d700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8cc100d70 0x7fe8cc19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8ca59c700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc0ff1e0 0x7fe8cc069130 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00efc0 tx=0x7fe8bc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8c37fe700 1 -- 172.21.15.93:0/376971710 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc018070 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8c37fe700 1 -- 172.21.15.93:0/376971710 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe8bc00f040 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8cc10ab30 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8c37fe700 1 -- 172.21.15.93:0/376971710 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8bc004310 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.701+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8cc10a9a0 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.706 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.702+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8cc19e4b0 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.703+0000 7fe8c37fe700 1 -- 172.21.15.93:0/376971710 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fe8bc00ca70 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.704+0000 7fe8c37fe700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe8b80a11e0 0x7fe8b80a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:00.707 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.704+0000 7fe8c9d9b700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe8b80a11e0 0x7fe8b80a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:00.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.704+0000 7fe8c37fe700 1 -- 172.21.15.93:0/376971710 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe8bc0cccd0 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.708 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.704+0000 7fe8c9d9b700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe8b80a11e0 0x7fe8b80a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc069690 tx=0x7fe8b400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:00.709 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.708+0000 7fe8c37fe700 1 -- 172.21.15.93:0/376971710 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe8bc096040 con 0x7fe8cc0ff1e0 2023-12-14T18:26:00.909 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.908+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8cc19eb20 con 0x7fe8b80a11e0 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.921+0000 7fe8c37fe700 1 -- 172.21.15.93:0/376971710 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+4185 (secure 0 0 0) 0x7fe8cc19eb20 con 0x7fe8b80a11e0 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stdout:mds.a smithi093 running (91s) 24s ago 91s 15.0M - 18.0.0-3905-g1faa3533 0caf899250bd dcb4bdf90ea4 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stdout:mds.b smithi097 running (90s) 25s ago 89s 30.0M - 18.0.0-3905-g1faa3533 0caf899250bd 979b27db1e77 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stdout:mds.c smithi160 running (98s) 24s ago 98s 19.7M - 18.0.0-3905-g1faa3533 0caf899250bd 6168261b32e6 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stdout:mds.d smithi093 running (104s) 24s ago 104s 14.7M - 18.0.0-3905-g1faa3533 0caf899250bd e1e62891400c 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stdout:mds.e smithi097 running (102s) 25s ago 102s 18.8M - 18.0.0-3905-g1faa3533 0caf899250bd 611c47ccc61f 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stdout:mds.f smithi160 running (100s) 24s ago 100s 17.2M - 18.0.0-3905-g1faa3533 0caf899250bd 345a5a56e376 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stdout:mds.g smithi093 running (93s) 24s ago 93s 14.2M - 18.0.0-3905-g1faa3533 0caf899250bd 77cf8a4eb84c 2023-12-14T18:26:00.922 INFO:teuthology.orchestra.run.smithi093.stdout:mds.h smithi097 running (97s) 25s ago 96s 18.6M - 18.0.0-3905-g1faa3533 0caf899250bd 2a637fbc3368 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mds.i smithi160 running (95s) 24s ago 95s 18.9M - 18.0.0-3905-g1faa3533 0caf899250bd f743a390e849 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mds.j smithi093 running (106s) 24s ago 105s 15.0M - 18.0.0-3905-g1faa3533 0caf899250bd a5483b8f4310 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mds.k smithi097 running (107s) 25s ago 107s 14.1M - 18.0.0-3905-g1faa3533 0caf899250bd 8b10cd3c5155 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mds.l smithi160 running (109s) 24s ago 109s 17.3M - 18.0.0-3905-g1faa3533 0caf899250bd 6b04e95430d3 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mgr.x smithi093 *:9283,8765 running (7m) 24s ago 7m 515M - 18.0.0-3905-g1faa3533 0caf899250bd 0e2a0b04bdb9 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mgr.y smithi097 *:8443,8765 running (5m) 25s ago 5m 447M - 18.0.0-3905-g1faa3533 0caf899250bd a97c760b82c0 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mgr.z smithi160 *:8443,8765 running (5m) 24s ago 5m 445M - 18.0.0-3905-g1faa3533 0caf899250bd 9a37f4fb58a6 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mon.a smithi093 running (7m) 24s ago 7m 92.9M 2048M 18.0.0-3905-g1faa3533 0caf899250bd 7331f67f21bf 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mon.b smithi097 running (5m) 25s ago 5m 67.7M 2048M 18.0.0-3905-g1faa3533 0caf899250bd a433b41687ff 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:mon.c smithi160 running (5m) 24s ago 5m 61.7M 2048M 18.0.0-3905-g1faa3533 0caf899250bd da39c0c8238f 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:osd.0 smithi093 running (5m) 24s ago 5m 68.2M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 2bec2b9b32f8 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:osd.1 smithi097 running (5m) 25s ago 5m 73.9M 4278M 18.0.0-3905-g1faa3533 0caf899250bd b4d06d65db3b 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:osd.2 smithi160 running (4m) 24s ago 4m 72.0M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 6a4d44bab771 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:osd.3 smithi093 running (4m) 24s ago 4m 66.3M 4278M 18.0.0-3905-g1faa3533 0caf899250bd b52dfb4d019d 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:osd.4 smithi097 running (4m) 25s ago 4m 60.0M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 0ee249dccb49 2023-12-14T18:26:00.923 INFO:teuthology.orchestra.run.smithi093.stdout:osd.5 smithi160 running (3m) 24s ago 3m 65.4M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 23a894055ad7 2023-12-14T18:26:00.924 INFO:teuthology.orchestra.run.smithi093.stdout:osd.6 smithi093 running (3m) 24s ago 3m 58.3M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 0cd0c5063993 2023-12-14T18:26:00.924 INFO:teuthology.orchestra.run.smithi093.stdout:osd.7 smithi097 running (3m) 25s ago 3m 63.4M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 5a5e2c228b74 2023-12-14T18:26:00.924 INFO:teuthology.orchestra.run.smithi093.stdout:osd.8 smithi160 running (2m) 24s ago 2m 59.7M 4278M 18.0.0-3905-g1faa3533 0caf899250bd d2e0f88e33c2 2023-12-14T18:26:00.924 INFO:teuthology.orchestra.run.smithi093.stdout:osd.9 smithi093 running (2m) 24s ago 2m 60.9M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 1893946be323 2023-12-14T18:26:00.924 INFO:teuthology.orchestra.run.smithi093.stdout:osd.10 smithi097 running (2m) 25s ago 2m 63.2M 4278M 18.0.0-3905-g1faa3533 0caf899250bd 1cf60ba2a26c 2023-12-14T18:26:00.924 INFO:teuthology.orchestra.run.smithi093.stdout:osd.11 smithi160 running (2m) 24s ago 2m 54.1M 4278M 18.0.0-3905-g1faa3533 0caf899250bd bf071a5527d1 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe8b80a11e0 msgr2=0x7fe8b80a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe8b80a11e0 0x7fe8b80a3690 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc069690 tx=0x7fe8b400b410 comp rx=0 tx=0).stop 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc0ff1e0 msgr2=0x7fe8cc069130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc0ff1e0 0x7fe8cc069130 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7fe8bc00efc0 tx=0x7fe8bc00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 shutdown_connections 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe8b80a11e0 0x7fe8b80a3690 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe8cc0ff1e0 0x7fe8cc069130 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe8cc100d70 0x7fe8cc19d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 --2- 172.21.15.93:0/376971710 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe8cc1003e0 0x7fe8cc19ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 >> 172.21.15.93:0/376971710 conn(0x7fe8cc0fa7f0 msgr2=0x7fe8cc0fcae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 shutdown_connections 2023-12-14T18:26:00.925 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:00.923+0000 7fe8d0d93700 1 -- 172.21.15.93:0/376971710 wait complete. 2023-12-14T18:26:01.226 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:01 smithi093 ceph-mon[42070]: pgmap v268: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:26:01.293 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch ls' 2023-12-14T18:26:01.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:01 smithi097 ceph-mon[43486]: pgmap v268: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:26:01.495 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:01 smithi160 ceph-mon[43840]: pgmap v268: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:26:01.607 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:02.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:02 smithi093 ceph-mon[42070]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:02.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:02 smithi097 ceph-mon[43486]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:02.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:02 smithi160 ceph-mon[43840]: from='client.15279 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:02.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 -- 172.21.15.93:0/3142920115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 msgr2=0x7f511c107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:02.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 --2- 172.21.15.93:0/3142920115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 0x7f511c107630 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f511400b0a0 tx=0x7f511400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 -- 172.21.15.93:0/3142920115 shutdown_connections 2023-12-14T18:26:02.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 --2- 172.21.15.93:0/3142920115 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 0x7f511c107630 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 --2- 172.21.15.93:0/3142920115 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0ff1e0 0x7f511c0ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 --2- 172.21.15.93:0/3142920115 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c1003e0 0x7f511c100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 -- 172.21.15.93:0/3142920115 >> 172.21.15.93:0/3142920115 conn(0x7f511c0fa7f0 msgr2=0x7f511c0fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:02.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 -- 172.21.15.93:0/3142920115 shutdown_connections 2023-12-14T18:26:02.683 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 -- 172.21.15.93:0/3142920115 wait complete. 2023-12-14T18:26:02.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 Processor -- start 2023-12-14T18:26:02.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.682+0000 7f5123640700 1 -- start start 2023-12-14T18:26:02.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5123640700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0ff1e0 0x7f511c198350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:02.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5123640700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c1003e0 0x7f511c198890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:02.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5123640700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 0x7f511c19f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:02.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5123640700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c19fee0 con 0x7f511c100d70 2023-12-14T18:26:02.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5123640700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c1a0020 con 0x7f511c0ff1e0 2023-12-14T18:26:02.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5123640700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f511c1a01b0 con 0x7f511c1003e0 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5121bdd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 0x7f511c19f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5121bdd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 0x7f511c19f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37190/0 (socket says 172.21.15.93:37190) 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5121bdd700 1 -- 172.21.15.93:0/1752706024 learned_addr learned my addr 172.21.15.93:0/1752706024 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f51213dc700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0ff1e0 0x7f511c198350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5120bdb700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c1003e0 0x7f511c198890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5121bdd700 1 -- 172.21.15.93:0/1752706024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c1003e0 msgr2=0x7f511c198890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5121bdd700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c1003e0 0x7f511c198890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5121bdd700 1 -- 172.21.15.93:0/1752706024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0ff1e0 msgr2=0x7f511c198350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5121bdd700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0ff1e0 0x7f511c198350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5121bdd700 1 -- 172.21.15.93:0/1752706024 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114009d00 con 0x7f511c100d70 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5121bdd700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 0x7f511c19f910 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f511400b070 tx=0x7f5114007990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f51127fc700 1 -- 172.21.15.93:0/1752706024 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f511400e040 con 0x7f511c100d70 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f511c10ab30 con 0x7f511c100d70 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f51127fc700 1 -- 172.21.15.93:0/1752706024 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5114004c50 con 0x7f511c100d70 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f51127fc700 1 -- 172.21.15.93:0/1752706024 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51140052a0 con 0x7f511c100d70 2023-12-14T18:26:02.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.683+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f511c10a9a0 con 0x7f511c100d70 2023-12-14T18:26:02.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.684+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511c1050c0 con 0x7f511c100d70 2023-12-14T18:26:02.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.685+0000 7f51127fc700 1 -- 172.21.15.93:0/1752706024 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7f511401d470 con 0x7f511c100d70 2023-12-14T18:26:02.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.686+0000 7f51127fc700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51080a1260 0x7f51080a3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:02.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.686+0000 7f51213dc700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51080a1260 0x7f51080a3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:02.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.686+0000 7f51127fc700 1 -- 172.21.15.93:0/1752706024 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f51140ceae0 con 0x7f511c100d70 2023-12-14T18:26:02.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.686+0000 7f51213dc700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51080a1260 0x7f51080a3710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f511c100240 tx=0x7f5118009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:02.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.689+0000 7f51127fc700 1 -- 172.21.15.93:0/1752706024 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5114097f10 con 0x7f511c100d70 2023-12-14T18:26:02.890 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.889+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f511c061b00 con 0x7f51080a1260 2023-12-14T18:26:02.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.894+0000 7f51127fc700 1 -- 172.21.15.93:0/1752706024 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+985 (secure 0 0 0) 0x7f511c061b00 con 0x7f51080a1260 2023-12-14T18:26:02.895 INFO:teuthology.orchestra.run.smithi093.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2023-12-14T18:26:02.895 INFO:teuthology.orchestra.run.smithi093.stdout:mds.all 12/12 27s ago 113s smithi093=a;smithi093=d;smithi093=g;smithi093=j;smithi097=b;smithi097=e;smithi097=h;smithi097=k;smithi160=c;smithi160=f;smithi160=i;smithi160=l;count:12 2023-12-14T18:26:02.895 INFO:teuthology.orchestra.run.smithi093.stdout:mgr 3/3 27s ago 5m smithi093=x;smithi097=y;smithi160=z;count:3 2023-12-14T18:26:02.895 INFO:teuthology.orchestra.run.smithi093.stdout:mon 3/3 27s ago 5m smithi093:172.21.15.93=a;smithi097:172.21.15.97=b;smithi160:172.21.15.160=c;count:3 2023-12-14T18:26:02.895 INFO:teuthology.orchestra.run.smithi093.stdout:osd 12 27s ago - 2023-12-14T18:26:02.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51080a1260 msgr2=0x7f51080a3710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:02.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51080a1260 0x7f51080a3710 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f511c100240 tx=0x7f5118009260 comp rx=0 tx=0).stop 2023-12-14T18:26:02.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 msgr2=0x7f511c19f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:02.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 0x7f511c19f910 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f511400b070 tx=0x7f5114007990 comp rx=0 tx=0).stop 2023-12-14T18:26:02.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 shutdown_connections 2023-12-14T18:26:02.897 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51080a1260 0x7f51080a3710 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f511c100d70 0x7f511c19f910 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f511c0ff1e0 0x7f511c198350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 --2- 172.21.15.93:0/1752706024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f511c1003e0 0x7f511c198890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:02.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 >> 172.21.15.93:0/1752706024 conn(0x7f511c0fa7f0 msgr2=0x7f511c0fca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:02.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 shutdown_connections 2023-12-14T18:26:02.898 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:02.896+0000 7f5123640700 1 -- 172.21.15.93:0/1752706024 wait complete. 2023-12-14T18:26:03.214 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:03 smithi093 ceph-mon[42070]: pgmap v269: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:26:03.259 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch host ls' 2023-12-14T18:26:03.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:03 smithi097 ceph-mon[43486]: pgmap v269: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:26:03.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:03 smithi160 ceph-mon[43840]: pgmap v269: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:26:03.570 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:04.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:04 smithi093 ceph-mon[42070]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:04.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:04 smithi097 ceph-mon[43486]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:04.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:04 smithi160 ceph-mon[43840]: from='client.15285 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:04.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.669+0000 7fc44aaf6700 1 -- 172.21.15.93:0/162173719 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 msgr2=0x7fc4440ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:04.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.669+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/162173719 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 0x7fc4440ff5f0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fc438009990 tx=0x7fc438009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.669+0000 7fc44aaf6700 1 -- 172.21.15.93:0/162173719 shutdown_connections 2023-12-14T18:26:04.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.669+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/162173719 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 0x7fc4440ff5f0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.669+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/162173719 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc444100d70 0x7fc444107630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.669+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/162173719 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4441003e0 0x7fc444100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.669+0000 7fc44aaf6700 1 -- 172.21.15.93:0/162173719 >> 172.21.15.93:0/162173719 conn(0x7fc4440fa7f0 msgr2=0x7fc4440fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:04.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.669+0000 7fc44aaf6700 1 -- 172.21.15.93:0/162173719 shutdown_connections 2023-12-14T18:26:04.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.669+0000 7fc44aaf6700 1 -- 172.21.15.93:0/162173719 wait complete. 2023-12-14T18:26:04.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc44aaf6700 1 Processor -- start 2023-12-14T18:26:04.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc44aaf6700 1 -- start start 2023-12-14T18:26:04.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc44aaf6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 0x7fc444194e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:04.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc44aaf6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4441003e0 0x7fc4441953a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:04.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc44aaf6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc444100d70 0x7fc44419c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:04.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc44aaf6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44419c9f0 con 0x7fc4440ff1e0 2023-12-14T18:26:04.672 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc44aaf6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44419cb30 con 0x7fc444100d70 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc44aaf6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc44419ccc0 con 0x7fc4441003e0 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc448892700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 0x7fc444194e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc448892700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 0x7fc444194e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37204/0 (socket says 172.21.15.93:37204) 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc448892700 1 -- 172.21.15.93:0/3619264017 learned_addr learned my addr 172.21.15.93:0/3619264017 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.670+0000 7fc443fff700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4441003e0 0x7fc4441953a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc449093700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc444100d70 0x7fc44419c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc448892700 1 -- 172.21.15.93:0/3619264017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4441003e0 msgr2=0x7fc4441953a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc448892700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4441003e0 0x7fc4441953a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc448892700 1 -- 172.21.15.93:0/3619264017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc444100d70 msgr2=0x7fc44419c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc448892700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc444100d70 0x7fc44419c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc448892700 1 -- 172.21.15.93:0/3619264017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4380095f0 con 0x7fc4440ff1e0 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc449093700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc444100d70 0x7fc44419c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:04.673 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc448892700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 0x7fc444194e60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc43801a840 tx=0x7fc43801a920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:04.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc441ffb700 1 -- 172.21.15.93:0/3619264017 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc43801f070 con 0x7fc4440ff1e0 2023-12-14T18:26:04.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc44410ab30 con 0x7fc4440ff1e0 2023-12-14T18:26:04.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc441ffb700 1 -- 172.21.15.93:0/3619264017 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc43801acb0 con 0x7fc4440ff1e0 2023-12-14T18:26:04.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc441ffb700 1 -- 172.21.15.93:0/3619264017 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc438025680 con 0x7fc4440ff1e0 2023-12-14T18:26:04.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.671+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc44410a9a0 con 0x7fc4440ff1e0 2023-12-14T18:26:04.674 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.672+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc444068920 con 0x7fc4440ff1e0 2023-12-14T18:26:04.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.673+0000 7fc441ffb700 1 -- 172.21.15.93:0/3619264017 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fc438023070 con 0x7fc4440ff1e0 2023-12-14T18:26:04.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.674+0000 7fc441ffb700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc4340a11e0 0x7fc4340a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:04.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.674+0000 7fc441ffb700 1 -- 172.21.15.93:0/3619264017 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc4380d06c0 con 0x7fc4440ff1e0 2023-12-14T18:26:04.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.674+0000 7fc443fff700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc4340a11e0 0x7fc4340a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:04.675 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.674+0000 7fc443fff700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc4340a11e0 0x7fc4340a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc430005ea0 tx=0x7fc430005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:04.678 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.677+0000 7fc441ffb700 1 -- 172.21.15.93:0/3619264017 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc438099a30 con 0x7fc4440ff1e0 2023-12-14T18:26:04.875 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.875+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fc444060f30 con 0x7fc4340a11e0 2023-12-14T18:26:04.877 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.876+0000 7fc441ffb700 1 -- 172.21.15.93:0/3619264017 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+191 (secure 0 0 0) 0x7fc444060f30 con 0x7fc4340a11e0 2023-12-14T18:26:04.877 INFO:teuthology.orchestra.run.smithi093.stdout:HOST ADDR LABELS STATUS 2023-12-14T18:26:04.877 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093 172.21.15.93 2023-12-14T18:26:04.877 INFO:teuthology.orchestra.run.smithi093.stdout:smithi097 172.21.15.97 2023-12-14T18:26:04.877 INFO:teuthology.orchestra.run.smithi093.stdout:smithi160 172.21.15.160 2023-12-14T18:26:04.877 INFO:teuthology.orchestra.run.smithi093.stdout:3 hosts in cluster 2023-12-14T18:26:04.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc4340a11e0 msgr2=0x7fc4340a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:04.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc4340a11e0 0x7fc4340a3690 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fc430005ea0 tx=0x7fc430005e30 comp rx=0 tx=0).stop 2023-12-14T18:26:04.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 msgr2=0x7fc444194e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:04.879 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 0x7fc444194e60 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7fc43801a840 tx=0x7fc43801a920 comp rx=0 tx=0).stop 2023-12-14T18:26:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 shutdown_connections 2023-12-14T18:26:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc4340a11e0 0x7fc4340a3690 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc4440ff1e0 0x7fc444194e60 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc444100d70 0x7fc44419c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 --2- 172.21.15.93:0/3619264017 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc4441003e0 0x7fc4441953a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.878+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 >> 172.21.15.93:0/3619264017 conn(0x7fc4440fa7f0 msgr2=0x7fc4440fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.879+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 shutdown_connections 2023-12-14T18:26:04.880 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:04.879+0000 7fc44aaf6700 1 -- 172.21.15.93:0/3619264017 wait complete. 2023-12-14T18:26:05.300 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:05 smithi093 ceph-mon[42070]: pgmap v270: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:26:05.320 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph orch device ls' 2023-12-14T18:26:05.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:05 smithi097 ceph-mon[43486]: pgmap v270: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:26:05.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:05 smithi160 ceph-mon[43840]: pgmap v270: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:26:05.633 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:06.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:06 smithi097 ceph-mon[43486]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:06.480 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:06 smithi093 ceph-mon[42070]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:06.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:06 smithi160 ceph-mon[43840]: from='client.15291 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:06.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.657+0000 7fbb0f847700 1 -- 172.21.15.93:0/2356273940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08103720 msgr2=0x7fbb08103b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.657+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2356273940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08103720 0x7fbb08103b70 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7fbb04009990 tx=0x7fbb04009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.657+0000 7fbb0f847700 1 -- 172.21.15.93:0/2356273940 shutdown_connections 2023-12-14T18:26:06.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.657+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2356273940 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb081040b0 0x7fbb0810a970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.659 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.657+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2356273940 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb08102520 0x7fbb08102930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.657+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2356273940 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08103720 0x7fbb08103b70 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.657+0000 7fbb0f847700 1 -- 172.21.15.93:0/2356273940 >> 172.21.15.93:0/2356273940 conn(0x7fbb080fdb90 msgr2=0x7fbb080fffa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:06.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.657+0000 7fbb0f847700 1 -- 172.21.15.93:0/2356273940 shutdown_connections 2023-12-14T18:26:06.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.657+0000 7fbb0f847700 1 -- 172.21.15.93:0/2356273940 wait complete. 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0f847700 1 Processor -- start 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0f847700 1 -- start start 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0f847700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08102520 0x7fbb081981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0f847700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb08103720 0x7fbb08198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0f847700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb081040b0 0x7fbb0819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0f847700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0819fd60 con 0x7fbb08102520 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0f847700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb0819fea0 con 0x7fbb081040b0 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0f847700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb081a0030 con 0x7fbb08103720 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0d5e3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08102520 0x7fbb081981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0d5e3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08102520 0x7fbb081981d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37224/0 (socket says 172.21.15.93:37224) 2023-12-14T18:26:06.662 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0d5e3700 1 -- 172.21.15.93:0/2843747536 learned_addr learned my addr 172.21.15.93:0/2843747536 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0cde2700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb08103720 0x7fbb08198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0dde4700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb081040b0 0x7fbb0819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0d5e3700 1 -- 172.21.15.93:0/2843747536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb08103720 msgr2=0x7fbb08198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0d5e3700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb08103720 0x7fbb08198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0d5e3700 1 -- 172.21.15.93:0/2843747536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb081040b0 msgr2=0x7fbb0819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0d5e3700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb081040b0 0x7fbb0819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.658+0000 7fbb0d5e3700 1 -- 172.21.15.93:0/2843747536 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb040095f0 con 0x7fbb08102520 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.659+0000 7fbb0d5e3700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08102520 0x7fbb081981d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800d9b0 tx=0x7fbaf800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.659+0000 7fbafe7fc700 1 -- 172.21.15.93:0/2843747536 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf8004020 con 0x7fbb08102520 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.659+0000 7fbafe7fc700 1 -- 172.21.15.93:0/2843747536 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbaf8009730 con 0x7fbb08102520 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.659+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb0810de70 con 0x7fbb08102520 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.659+0000 7fbafe7fc700 1 -- 172.21.15.93:0/2843747536 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbaf80039e0 con 0x7fbb08102520 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.659+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb0810dce0 con 0x7fbb08102520 2023-12-14T18:26:06.663 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.660+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb08109ac0 con 0x7fbb08102520 2023-12-14T18:26:06.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.661+0000 7fbafe7fc700 1 -- 172.21.15.93:0/2843747536 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 17) v1 ==== 135766+0+0 (secure 0 0 0) 0x7fbaf80098a0 con 0x7fbb08102520 2023-12-14T18:26:06.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.661+0000 7fbafe7fc700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbaf40a11e0 0x7fbaf40a3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:06.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.661+0000 7fbb0cde2700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbaf40a11e0 0x7fbaf40a3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:06.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.661+0000 7fbafe7fc700 1 -- 172.21.15.93:0/2843747536 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbaf80cca00 con 0x7fbb08102520 2023-12-14T18:26:06.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.662+0000 7fbb0cde2700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbaf40a11e0 0x7fbaf40a3690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400b2e0 tx=0x7fbb04005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:06.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.666+0000 7fbafe7fc700 1 -- 172.21.15.93:0/2843747536 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbaf8095ce0 con 0x7fbb08102520 2023-12-14T18:26:06.864 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.862+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb08061a30 con 0x7fbaf40a11e0 2023-12-14T18:26:06.866 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.864+0000 7fbafe7fc700 1 -- 172.21.15.93:0/2843747536 <== mgr.14152 v2:172.21.15.93:6800/3802499407 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+492 (secure 0 0 0) 0x7fbb08061a30 con 0x7fbaf40a11e0 2023-12-14T18:26:06.866 INFO:teuthology.orchestra.run.smithi093.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2023-12-14T18:26:06.866 INFO:teuthology.orchestra.run.smithi093.stdout:smithi093 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT62040150400BGN 372G No 2m ago LVM detected, locked 2023-12-14T18:26:06.866 INFO:teuthology.orchestra.run.smithi093.stdout:smithi097 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT6204013T400BGN 372G No 2m ago LVM detected, locked 2023-12-14T18:26:06.866 INFO:teuthology.orchestra.run.smithi093.stdout:smithi160 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT623300JY400BGN 372G No 2m ago LVM detected, locked 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbaf40a11e0 msgr2=0x7fbaf40a3690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbaf40a11e0 0x7fbaf40a3690 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fbb0400b2e0 tx=0x7fbb04005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08102520 msgr2=0x7fbb081981d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08102520 0x7fbb081981d0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fbaf800d9b0 tx=0x7fbaf800dcc0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 shutdown_connections 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbaf40a11e0 0x7fbaf40a3690 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbb08102520 0x7fbb081981d0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbb081040b0 0x7fbb0819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 --2- 172.21.15.93:0/2843747536 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbb08103720 0x7fbb08198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:06.868 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 >> 172.21.15.93:0/2843747536 conn(0x7fbb080fdb90 msgr2=0x7fbb08106c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:06.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 shutdown_connections 2023-12-14T18:26:06.869 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:06.866+0000 7fbb0f847700 1 -- 172.21.15.93:0/2843747536 wait complete. 2023-12-14T18:26:07.170 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:07 smithi093 ceph-mon[42070]: pgmap v271: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:26:07.243 INFO:teuthology.run_tasks:Running task cephadm.shell... 2023-12-14T18:26:07.253 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:26:07.253 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph fs dump' 2023-12-14T18:26:07.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:07 smithi097 ceph-mon[43486]: pgmap v271: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:26:07.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:07 smithi160 ceph-mon[43840]: pgmap v271: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 5 op/s 2023-12-14T18:26:07.565 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:08.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:08 smithi093 ceph-mon[42070]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:08.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:08 smithi093 ceph-mon[42070]: mgrmap e18: x(active, since 6m), standbys: z, y 2023-12-14T18:26:08.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:08 smithi097 ceph-mon[43486]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:08.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:08 smithi097 ceph-mon[43486]: mgrmap e18: x(active, since 6m), standbys: z, y 2023-12-14T18:26:08.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:08 smithi160 ceph-mon[43840]: from='client.15297 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:08.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:08 smithi160 ceph-mon[43840]: mgrmap e18: x(active, since 6m), standbys: z, y 2023-12-14T18:26:08.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.605+0000 7fc3fb029700 1 -- 172.21.15.93:0/2086203753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 msgr2=0x7fc3f4107630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.605+0000 7fc3fb029700 1 --2- 172.21.15.93:0/2086203753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 0x7fc3f4107630 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009f90 tx=0x7fc3ec00b330 comp rx=0 tx=0).stop 2023-12-14T18:26:08.607 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.605+0000 7fc3fb029700 1 -- 172.21.15.93:0/2086203753 shutdown_connections 2023-12-14T18:26:08.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.605+0000 7fc3fb029700 1 --2- 172.21.15.93:0/2086203753 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 0x7fc3f4107630 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.605+0000 7fc3fb029700 1 --2- 172.21.15.93:0/2086203753 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3f40ff1e0 0x7fc3f40ff5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.605+0000 7fc3fb029700 1 --2- 172.21.15.93:0/2086203753 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f41003e0 0x7fc3f4100830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.605+0000 7fc3fb029700 1 -- 172.21.15.93:0/2086203753 >> 172.21.15.93:0/2086203753 conn(0x7fc3f40fa7f0 msgr2=0x7fc3f40fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:08.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.605+0000 7fc3fb029700 1 -- 172.21.15.93:0/2086203753 shutdown_connections 2023-12-14T18:26:08.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.605+0000 7fc3fb029700 1 -- 172.21.15.93:0/2086203753 wait complete. 2023-12-14T18:26:08.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3fb029700 1 Processor -- start 2023-12-14T18:26:08.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3fb029700 1 -- start start 2023-12-14T18:26:08.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3fb029700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3f40ff1e0 0x7fc3f4069110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3fb029700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f41003e0 0x7fc3f419d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3fb029700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 0x7fc3f419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3fb029700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f419feb0 con 0x7fc3f4100d70 2023-12-14T18:26:08.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3fb029700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41a0020 con 0x7fc3f41003e0 2023-12-14T18:26:08.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3fb029700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f41a01e0 con 0x7fc3f40ff1e0 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3f95c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 0x7fc3f419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3f8dc5700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3f40ff1e0 0x7fc3f4069110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3f95c6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 0x7fc3f419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37236/0 (socket says 172.21.15.93:37236) 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3f95c6700 1 -- 172.21.15.93:0/1419089969 learned_addr learned my addr 172.21.15.93:0/1419089969 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.606+0000 7fc3f3fff700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f41003e0 0x7fc3f419d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3f95c6700 1 -- 172.21.15.93:0/1419089969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3f40ff1e0 msgr2=0x7fc3f4069110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3f95c6700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3f40ff1e0 0x7fc3f4069110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3f95c6700 1 -- 172.21.15.93:0/1419089969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f41003e0 msgr2=0x7fc3f419d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3f95c6700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f41003e0 0x7fc3f419d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3f95c6700 1 -- 172.21.15.93:0/1419089969 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3ec009c40 con 0x7fc3f4100d70 2023-12-14T18:26:08.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3f95c6700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 0x7fc3f419f800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00b6c0 tx=0x7fc3ec004ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:08.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3f1ffb700 1 -- 172.21.15.93:0/1419089969 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec00e040 con 0x7fc3f4100d70 2023-12-14T18:26:08.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3f1ffb700 1 -- 172.21.15.93:0/1419089969 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fc3ec00bbf0 con 0x7fc3f4100d70 2023-12-14T18:26:08.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3f1ffb700 1 -- 172.21.15.93:0/1419089969 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3ec0052a0 con 0x7fc3f4100d70 2023-12-14T18:26:08.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f410ab30 con 0x7fc3f4100d70 2023-12-14T18:26:08.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.607+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f410a9a0 con 0x7fc3f4100d70 2023-12-14T18:26:08.612 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.608+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f4066ea0 con 0x7fc3f4100d70 2023-12-14T18:26:08.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.609+0000 7fc3f1ffb700 1 -- 172.21.15.93:0/1419089969 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fc3ec01d470 con 0x7fc3f4100d70 2023-12-14T18:26:08.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.610+0000 7fc3f1ffb700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3e40a11d0 0x7fc3e40a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:08.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.610+0000 7fc3f1ffb700 1 -- 172.21.15.93:0/1419089969 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fc3ec0ce000 con 0x7fc3f4100d70 2023-12-14T18:26:08.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.610+0000 7fc3f8dc5700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3e40a11d0 0x7fc3e40a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:08.613 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.610+0000 7fc3f8dc5700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3e40a11d0 0x7fc3e40a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4100240 tx=0x7fc3e800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:08.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.614+0000 7fc3f1ffb700 1 -- 172.21.15.93:0/1419089969 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fc3ec0974d0 con 0x7fc3f4100d70 2023-12-14T18:26:08.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.845+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump"} v 0) v1 -- 0x7fc3f41060f0 con 0x7fc3f4100d70 2023-12-14T18:26:08.847 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.845+0000 7fc3f1ffb700 1 -- 172.21.15.93:0/1419089969 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 24 v24) v1 ==== 76+0+2895 (secure 0 0 0) 0x7fc3ec09aaf0 con 0x7fc3f4100d70 2023-12-14T18:26:08.847 INFO:teuthology.orchestra.run.smithi093.stdout:e24 2023-12-14T18:26:08.847 INFO:teuthology.orchestra.run.smithi093.stdout:enable_multiple, ever_enabled_multiple: 1,1 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout:legacy client fscid: 1 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout:Filesystem 'cephfs' (1) 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout:fs_name cephfs 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout:epoch 24 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout:created 2023-12-14T18:24:36.557875+0000 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout:modified 2023-12-14T18:25:01.760987+0000 2023-12-14T18:26:08.848 INFO:teuthology.orchestra.run.smithi093.stdout:tableserver 0 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:root 0 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:session_timeout 300 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:session_autoclose 300 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:max_file_size 1099511627776 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:max_xattr_size 65536 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:required_client_features {} 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:last_failure 0 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:last_failure_osd_epoch 0 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2} 2023-12-14T18:26:08.849 INFO:teuthology.orchestra.run.smithi093.stdout:max_mds 3 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:in 0,1,2 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:up {0=24461,1=24442,2=24422} 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:failed 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:damaged 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:stopped 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:data_pools [3,4] 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:metadata_pool 2 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:inline_data disabled 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:balancer 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:bal_rank_mask -1 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:standby_count_wanted 0 2023-12-14T18:26:08.850 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.b{0:24461} state up:active seq 3 addr [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.h{0:24440} state up:standby-replay seq 1 addr [v2:172.21.15.97:6836/3050538896,v1:172.21.15.97:6837/3050538896] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.i{1:24442} state up:active seq 10 addr [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.c{1:24424} state up:standby-replay seq 1 addr [v2:172.21.15.160:6836/233011976,v1:172.21.15.160:6837/233011976] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.e{2:24422} state up:active seq 12 addr [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.f{2:24415} state up:standby-replay seq 1 addr [v2:172.21.15.160:6834/2285100381,v1:172.21.15.160:6835/2285100381] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout:Standby daemons: 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.j{-1:14550} state up:standby seq 1 addr [v2:172.21.15.93:6834/53835883,v1:172.21.15.93:6835/53835883] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.851 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.d{-1:14562} state up:standby seq 1 addr [v2:172.21.15.93:6836/1107020110,v1:172.21.15.93:6837/1107020110] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.852 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.g{-1:14607} state up:standby seq 1 addr [v2:172.21.15.93:6838/1450887105,v1:172.21.15.93:6839/1450887105] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.852 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.a{-1:14619} state up:standby seq 1 addr [v2:172.21.15.93:6840/4265710188,v1:172.21.15.93:6841/4265710188] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.852 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.l{-1:24385} state up:standby seq 1 addr [v2:172.21.15.160:6832/2184947052,v1:172.21.15.160:6833/2184947052] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.852 INFO:teuthology.orchestra.run.smithi093.stdout:[mds.k{-1:24398} state up:standby seq 1 addr [v2:172.21.15.97:6832/1737809375,v1:172.21.15.97:6833/1737809375] compat {c=[1],r=[1],i=[7ff]}] 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3e40a11d0 msgr2=0x7fc3e40a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3e40a11d0 0x7fc3e40a3680 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4100240 tx=0x7fc3e800b410 comp rx=0 tx=0).stop 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 msgr2=0x7fc3f419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 0x7fc3f419f800 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec00b6c0 tx=0x7fc3ec004ad0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 shutdown_connections 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fc3e40a11d0 0x7fc3e40a3680 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fc3f4100d70 0x7fc3f419f800 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fc3f40ff1e0 0x7fc3f4069110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 --2- 172.21.15.93:0/1419089969 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fc3f41003e0 0x7fc3f419d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:08.853 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 >> 172.21.15.93:0/1419089969 conn(0x7fc3f40fa7f0 msgr2=0x7fc3f40fcaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:08.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 shutdown_connections 2023-12-14T18:26:08.854 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:08.847+0000 7fc3fb029700 1 -- 172.21.15.93:0/1419089969 wait complete. 2023-12-14T18:26:08.854 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 24 2023-12-14T18:26:09.166 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:09 smithi093 ceph-mon[42070]: pgmap v272: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:26:09.167 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:09 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1419089969' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-14T18:26:09.225 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- bash -c 'ceph osd dump' 2023-12-14T18:26:09.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:09 smithi097 ceph-mon[43486]: pgmap v272: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:26:09.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:09 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1419089969' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-14T18:26:09.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:09 smithi160 ceph-mon[43840]: pgmap v272: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:26:09.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:09 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1419089969' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2023-12-14T18:26:09.536 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:10.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.621+0000 7fb610c75700 1 -- 172.21.15.93:0/856554826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c1002f0 msgr2=0x7fb60c100700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.621+0000 7fb610c75700 1 --2- 172.21.15.93:0/856554826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c1002f0 0x7fb60c100700 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7fb600009990 tx=0x7fb600009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.625 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.621+0000 7fb610c75700 1 -- 172.21.15.93:0/856554826 shutdown_connections 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.621+0000 7fb610c75700 1 --2- 172.21.15.93:0/856554826 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c1002f0 0x7fb60c100700 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.621+0000 7fb610c75700 1 --2- 172.21.15.93:0/856554826 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb60c101e80 0x7fb60c108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.621+0000 7fb610c75700 1 --2- 172.21.15.93:0/856554826 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60c1014f0 0x7fb60c101940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.621+0000 7fb610c75700 1 -- 172.21.15.93:0/856554826 >> 172.21.15.93:0/856554826 conn(0x7fb60c0fb900 msgr2=0x7fb60c0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.621+0000 7fb610c75700 1 -- 172.21.15.93:0/856554826 shutdown_connections 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.621+0000 7fb610c75700 1 -- 172.21.15.93:0/856554826 wait complete. 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb610c75700 1 Processor -- start 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb610c75700 1 -- start start 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb610c75700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb60c1002f0 0x7fb60c1960e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb610c75700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60c1014f0 0x7fb60c196620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb610c75700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c101e80 0x7fb60c19d6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb610c75700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c19dc70 con 0x7fb60c101e80 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb610c75700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c19ddb0 con 0x7fb60c1014f0 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb610c75700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb60c19df40 con 0x7fb60c1002f0 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb60ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c101e80 0x7fb60c19d6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb60ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c101e80 0x7fb60c19d6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37252/0 (socket says 172.21.15.93:37252) 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb60ad9d700 1 -- 172.21.15.93:0/3696651311 learned_addr learned my addr 172.21.15.93:0/3696651311 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb60a59c700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb60c1002f0 0x7fb60c1960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb609d9b700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60c1014f0 0x7fb60c196620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb60ad9d700 1 -- 172.21.15.93:0/3696651311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb60c1002f0 msgr2=0x7fb60c1960e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb60ad9d700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb60c1002f0 0x7fb60c1960e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb60ad9d700 1 -- 172.21.15.93:0/3696651311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60c1014f0 msgr2=0x7fb60c196620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb60ad9d700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60c1014f0 0x7fb60c196620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.622+0000 7fb60ad9d700 1 -- 172.21.15.93:0/3696651311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6000095f0 con 0x7fb60c101e80 2023-12-14T18:26:10.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.623+0000 7fb60ad9d700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c101e80 0x7fb60c19d6a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb60400b9a0 tx=0x7fb60400bcb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:10.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.623+0000 7fb5fb7fe700 1 -- 172.21.15.93:0/3696651311 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb60400d850 con 0x7fb60c101e80 2023-12-14T18:26:10.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.623+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb60c10bd40 con 0x7fb60c101e80 2023-12-14T18:26:10.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.623+0000 7fb5fb7fe700 1 -- 172.21.15.93:0/3696651311 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb60400de70 con 0x7fb60c101e80 2023-12-14T18:26:10.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.623+0000 7fb5fb7fe700 1 -- 172.21.15.93:0/3696651311 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb604017400 con 0x7fb60c101e80 2023-12-14T18:26:10.628 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.623+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb60c10bbb0 con 0x7fb60c101e80 2023-12-14T18:26:10.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.624+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb60c19e700 con 0x7fb60c101e80 2023-12-14T18:26:10.629 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.625+0000 7fb5fb7fe700 1 -- 172.21.15.93:0/3696651311 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fb60400e600 con 0x7fb60c101e80 2023-12-14T18:26:10.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.625+0000 7fb5fb7fe700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb5f40a1150 0x7fb5f40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:10.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.626+0000 7fb60a59c700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb5f40a1150 0x7fb5f40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:10.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.626+0000 7fb5fb7fe700 1 -- 172.21.15.93:0/3696651311 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fb604006e40 con 0x7fb60c101e80 2023-12-14T18:26:10.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.626+0000 7fb60a59c700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb5f40a1150 0x7fb5f40a3600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb600006010 tx=0x7fb600005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:10.631 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.629+0000 7fb5fb7fe700 1 -- 172.21.15.93:0/3696651311 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb604095db0 con 0x7fb60c101e80 2023-12-14T18:26:10.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.819+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump"} v 0) v1 -- 0x7fb60c107240 con 0x7fb60c101e80 2023-12-14T18:26:10.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.820+0000 7fb5fb7fe700 1 -- 172.21.15.93:0/3696651311 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v86) v1 ==== 56+0+5575 (secure 0 0 0) 0x7fb6040993d0 con 0x7fb60c101e80 2023-12-14T18:26:10.822 INFO:teuthology.orchestra.run.smithi093.stdout:epoch 86 2023-12-14T18:26:10.822 INFO:teuthology.orchestra.run.smithi093.stdout:fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 2023-12-14T18:26:10.822 INFO:teuthology.orchestra.run.smithi093.stdout:created 2023-12-14T18:18:47.086099+0000 2023-12-14T18:26:10.822 INFO:teuthology.orchestra.run.smithi093.stdout:modified 2023-12-14T18:25:37.836200+0000 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi093.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi093.stdout:crush_version 27 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi093.stdout:full_ratio 0.95 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi093.stdout:backfillfull_ratio 0.9 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi093.stdout:nearfull_ratio 0.85 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi093.stdout:require_min_compat_client luminous 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi093.stdout:min_compat_client luminous 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi093.stdout:require_osd_release reef 2023-12-14T18:26:10.823 INFO:teuthology.orchestra.run.smithi093.stdout:stretch_mode_enabled false 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 2.25 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 80 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:max_osd 12 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6802/822854131,v1:172.21.15.93:6803/822854131] [v2:172.21.15.93:6804/822854131,v1:172.21.15.93:6805/822854131] exists,up f4bece78-53dc-4e27-bb2b-32a16baa8936 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6800/3276231390,v1:172.21.15.97:6801/3276231390] [v2:172.21.15.97:6802/3276231390,v1:172.21.15.97:6803/3276231390] exists,up a2968fe6-d55f-4ae4-a2bc-e06b379812ae 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:osd.2 up in weight 1 up_from 18 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6800/3346159121,v1:172.21.15.160:6801/3346159121] [v2:172.21.15.160:6802/3346159121,v1:172.21.15.160:6803/3346159121] exists,up a9138f66-3120-4842-af4c-40a6c4c57710 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:osd.3 up in weight 1 up_from 24 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6810/980379303,v1:172.21.15.93:6811/980379303] [v2:172.21.15.93:6812/980379303,v1:172.21.15.93:6813/980379303] exists,up 317f1a16-4f77-46e5-aa43-63e8d930682b 2023-12-14T18:26:10.824 INFO:teuthology.orchestra.run.smithi093.stdout:osd.4 up in weight 1 up_from 31 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6808/3095116912,v1:172.21.15.97:6809/3095116912] [v2:172.21.15.97:6810/3095116912,v1:172.21.15.97:6811/3095116912] exists,up 7afe6ea6-e5ce-4fea-8e55-a5fbcba54181 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:osd.5 up in weight 1 up_from 36 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6808/1570120079,v1:172.21.15.160:6809/1570120079] [v2:172.21.15.160:6810/1570120079,v1:172.21.15.160:6811/1570120079] exists,up 6882d38d-40ad-40d7-aeb3-8c1e9367ef16 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:osd.6 up in weight 1 up_from 43 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6818/848954134,v1:172.21.15.93:6819/848954134] [v2:172.21.15.93:6820/848954134,v1:172.21.15.93:6821/848954134] exists,up 4f046b70-3c19-4dac-99f8-2c72520268be 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:osd.7 up in weight 1 up_from 48 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6816/1426225102,v1:172.21.15.97:6817/1426225102] [v2:172.21.15.97:6818/1426225102,v1:172.21.15.97:6819/1426225102] exists,up 527a2a02-7c78-4572-9493-54af215f8c7e 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:osd.8 up in weight 1 up_from 55 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6816/3984975827,v1:172.21.15.160:6817/3984975827] [v2:172.21.15.160:6818/3984975827,v1:172.21.15.160:6819/3984975827] exists,up 5d79000b-a942-43b7-85f7-620e6b98bf90 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:osd.9 up in weight 1 up_from 62 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.93:6826/4200476017,v1:172.21.15.93:6827/4200476017] [v2:172.21.15.93:6828/4200476017,v1:172.21.15.93:6829/4200476017] exists,up 14e038ed-c6e1-4d19-88cc-329ffe2343e9 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:osd.10 up in weight 1 up_from 67 up_thru 78 down_at 0 last_clean_interval [0,0) [v2:172.21.15.97:6824/240921879,v1:172.21.15.97:6825/240921879] [v2:172.21.15.97:6826/240921879,v1:172.21.15.97:6827/240921879] exists,up 44df1de1-facd-4592-855a-d32c456cf8ee 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:osd.11 up in weight 1 up_from 75 up_thru 85 down_at 0 last_clean_interval [0,0) [v2:172.21.15.160:6824/747472238,v1:172.21.15.160:6825/747472238] [v2:172.21.15.160:6826/747472238,v1:172.21.15.160:6827/747472238] exists,up 82c3b552-1386-4b96-89c6-4263d10a3f2c 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 2.6 [8,11] 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 2.a [2,11] 2023-12-14T18:26:10.825 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 2.1b [2,4] 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.10 [5,9] 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.24 [4,9] 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 3.34 [4,9] 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 4.2 [9,1] 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 4.3 [5,8] 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 4.c [9,10] 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:pg_upmap_items 4.f [9,8] 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6800/2647964405 expires 2023-12-15T18:19:35.884432+0000 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/3363004642 expires 2023-12-15T18:19:35.884432+0000 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/2583563521 expires 2023-12-15T18:19:15.054604+0000 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/1226958737 expires 2023-12-15T18:19:35.884432+0000 2023-12-14T18:26:10.826 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6800/1555415520 expires 2023-12-15T18:19:15.054604+0000 2023-12-14T18:26:10.827 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/216269584 expires 2023-12-15T18:19:15.054604+0000 2023-12-14T18:26:10.827 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6801/1555415520 expires 2023-12-15T18:19:15.054604+0000 2023-12-14T18:26:10.827 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:6801/2647964405 expires 2023-12-15T18:19:35.884432+0000 2023-12-14T18:26:10.827 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/630684095 expires 2023-12-15T18:19:15.054604+0000 2023-12-14T18:26:10.827 INFO:teuthology.orchestra.run.smithi093.stdout:blocklist 172.21.15.93:0/2376464512 expires 2023-12-15T18:19:35.884432+0000 2023-12-14T18:26:10.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb5f40a1150 msgr2=0x7fb5f40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb5f40a1150 0x7fb5f40a3600 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb600006010 tx=0x7fb600005960 comp rx=0 tx=0).stop 2023-12-14T18:26:10.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c101e80 msgr2=0x7fb60c19d6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:10.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c101e80 0x7fb60c19d6a0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7fb60400b9a0 tx=0x7fb60400bcb0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 shutdown_connections 2023-12-14T18:26:10.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fb5f40a1150 0x7fb5f40a3600 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb60c101e80 0x7fb60c19d6a0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.828 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb60c1002f0 0x7fb60c1960e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 --2- 172.21.15.93:0/3696651311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb60c1014f0 0x7fb60c196620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:10.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 >> 172.21.15.93:0/3696651311 conn(0x7fb60c0fb900 msgr2=0x7fb60c0fdbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:10.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 shutdown_connections 2023-12-14T18:26:10.829 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:10.822+0000 7fb610c75700 1 -- 172.21.15.93:0/3696651311 wait complete. 2023-12-14T18:26:11.141 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:11 smithi093 ceph-mon[42070]: pgmap v273: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:26:11.141 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:11 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3696651311' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-14T18:26:11.201 INFO:teuthology.run_tasks:Running task fs.ready... 2023-12-14T18:26:11.213 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:11.340 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.338+0000 7f51756f8700 1 -- 172.21.15.93:0/1334572663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fed30 msgr2=0x7f51701088f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.338+0000 7f51756f8700 1 --2- 172.21.15.93:0/1334572663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fed30 0x7f51701088f0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f516800b0a0 tx=0x7f516800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.338+0000 7f51756f8700 1 -- 172.21.15.93:0/1334572663 shutdown_connections 2023-12-14T18:26:11.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.338+0000 7f51756f8700 1 --2- 172.21.15.93:0/1334572663 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fed30 0x7f51701088f0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.338+0000 7f51756f8700 1 --2- 172.21.15.93:0/1334572663 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5170101910 0x7f5170101d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.338+0000 7f51756f8700 1 --2- 172.21.15.93:0/1334572663 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f51700fe380 0x7f51700fe7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.338+0000 7f51756f8700 1 -- 172.21.15.93:0/1334572663 >> 172.21.15.93:0/1334572663 conn(0x7f51700f9fb0 msgr2=0x7f51700fc3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:11.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 -- 172.21.15.93:0/1334572663 shutdown_connections 2023-12-14T18:26:11.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 -- 172.21.15.93:0/1334572663 wait complete. 2023-12-14T18:26:11.341 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 Processor -- start 2023-12-14T18:26:11.344 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 -- start start 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fe380 0x7f5170196090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51700fed30 0x7f51701965d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5170101910 0x7f517019d640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517019dc10 con 0x7f51700fe380 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517019dd50 con 0x7f51700fed30 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.339+0000 7f51756f8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f517019dee0 con 0x7f5170101910 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fe380 0x7f5170196090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fe380 0x7f5170196090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46414/0 (socket says 172.21.15.93:46414) 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516effd700 1 -- 172.21.15.93:0/3237520275 learned_addr learned my addr 172.21.15.93:0/3237520275 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516f7fe700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5170101910 0x7f517019d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516e7fc700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51700fed30 0x7f51701965d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516effd700 1 -- 172.21.15.93:0/3237520275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5170101910 msgr2=0x7f517019d640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.345 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516effd700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5170101910 0x7f517019d640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516effd700 1 -- 172.21.15.93:0/3237520275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51700fed30 msgr2=0x7f51701965d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516effd700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51700fed30 0x7f51701965d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516effd700 1 -- 172.21.15.93:0/3237520275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5168009d00 con 0x7f51700fe380 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516e7fc700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51700fed30 0x7f51701965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f516effd700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fe380 0x7f5170196090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f516400d980 tx=0x7f516400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f5157fff700 1 -- 172.21.15.93:0/3237520275 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516400b950 con 0x7f51700fe380 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f5157fff700 1 -- 172.21.15.93:0/3237520275 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5164007620 con 0x7f51700fe380 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f517010bc80 con 0x7f51700fe380 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f5157fff700 1 -- 172.21.15.93:0/3237520275 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f516400f3c0 con 0x7f51700fe380 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.340+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f517010baf0 con 0x7f51700fe380 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.341+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5170107340 con 0x7f51700fe380 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.342+0000 7f5157fff700 1 -- 172.21.15.93:0/3237520275 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5164010640 con 0x7f51700fe380 2023-12-14T18:26:11.346 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.343+0000 7f5157fff700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51580a11d0 0x7f51580a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:11.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.343+0000 7f516e7fc700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51580a11d0 0x7f51580a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:11.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.343+0000 7f5157fff700 1 -- 172.21.15.93:0/3237520275 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f51640cd2f0 con 0x7f51700fe380 2023-12-14T18:26:11.347 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:11 smithi097 ceph-mon[43486]: pgmap v273: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:26:11.347 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:11 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3696651311' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-14T18:26:11.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.343+0000 7f516e7fc700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51580a11d0 0x7f51580a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5170068f80 tx=0x7f5160009670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:11.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.346+0000 7f5157fff700 1 -- 172.21.15.93:0/3237520275 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f51640966d0 con 0x7f51700fe380 2023-12-14T18:26:11.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:11 smithi160 ceph-mon[43840]: pgmap v273: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 426 B/s wr, 5 op/s 2023-12-14T18:26:11.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:11 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3696651311' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2023-12-14T18:26:11.540 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.538+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5170067e40 con 0x7f51700fe380 2023-12-14T18:26:11.542 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.540+0000 7f5157fff700 1 -- 172.21.15.93:0/3237520275 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24357 (secure 0 0 0) 0x7f516400ee60 con 0x7f51700fe380 2023-12-14T18:26:11.542 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:26:11.542 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":86,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:25:37.836200+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:11.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51580a11d0 msgr2=0x7f51580a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51580a11d0 0x7f51580a3680 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5170068f80 tx=0x7f5160009670 comp rx=0 tx=0).stop 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fe380 msgr2=0x7f5170196090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fe380 0x7f5170196090 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f516400d980 tx=0x7f516400dc90 comp rx=0 tx=0).stop 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 shutdown_connections 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f51580a11d0 0x7f51580a3680 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f51700fe380 0x7f5170196090 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5170101910 0x7f517019d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 --2- 172.21.15.93:0/3237520275 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f51700fed30 0x7f51701965d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.542+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 >> 172.21.15.93:0/3237520275 conn(0x7f51700f9fb0 msgr2=0x7f51700fc180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.543+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 shutdown_connections 2023-12-14T18:26:11.545 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.543+0000 7f51756f8700 1 -- 172.21.15.93:0/3237520275 wait complete. 2023-12-14T18:26:11.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:24:34.407045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:24:35.728548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:24:38.796189+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:26:11.558 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:26:11.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.681+0000 7fe733461700 1 -- 172.21.15.93:0/818247283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c100ca0 msgr2=0x7fe72c107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.681+0000 7fe733461700 1 --2- 172.21.15.93:0/818247283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c100ca0 0x7fe72c107560 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fe72400b0a0 tx=0x7fe72400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.681+0000 7fe733461700 1 -- 172.21.15.93:0/818247283 shutdown_connections 2023-12-14T18:26:11.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.681+0000 7fe733461700 1 --2- 172.21.15.93:0/818247283 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c100ca0 0x7fe72c107560 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.681+0000 7fe733461700 1 --2- 172.21.15.93:0/818247283 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72c0ff120 0x7fe72c0ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.681+0000 7fe733461700 1 --2- 172.21.15.93:0/818247283 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72c100310 0x7fe72c100760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.681+0000 7fe733461700 1 -- 172.21.15.93:0/818247283 >> 172.21.15.93:0/818247283 conn(0x7fe72c0fa740 msgr2=0x7fe72c0fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:11.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.681+0000 7fe733461700 1 -- 172.21.15.93:0/818247283 shutdown_connections 2023-12-14T18:26:11.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.681+0000 7fe733461700 1 -- 172.21.15.93:0/818247283 wait complete. 2023-12-14T18:26:11.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe733461700 1 Processor -- start 2023-12-14T18:26:11.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe733461700 1 -- start start 2023-12-14T18:26:11.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe733461700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c0ff120 0x7fe72c069b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:11.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe733461700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72c100310 0x7fe72c19ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:11.687 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe733461700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72c100ca0 0x7fe72c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe733461700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c19dbe0 con 0x7fe72c0ff120 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe733461700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c19dd50 con 0x7fe72c100310 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe733461700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe72c19df10 con 0x7fe72c100ca0 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe7311fd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c0ff120 0x7fe72c069b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe7311fd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c0ff120 0x7fe72c069b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46434/0 (socket says 172.21.15.93:46434) 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe7311fd700 1 -- 172.21.15.93:0/2948051824 learned_addr learned my addr 172.21.15.93:0/2948051824 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe7309fc700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72c100310 0x7fe72c19ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.682+0000 7fe7319fe700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72c100ca0 0x7fe72c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe7311fd700 1 -- 172.21.15.93:0/2948051824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72c100ca0 msgr2=0x7fe72c19d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe7311fd700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72c100ca0 0x7fe72c19d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe7311fd700 1 -- 172.21.15.93:0/2948051824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72c100310 msgr2=0x7fe72c19ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe7311fd700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72c100310 0x7fe72c19ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe7311fd700 1 -- 172.21.15.93:0/2948051824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe724009d00 con 0x7fe72c0ff120 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe7311fd700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c0ff120 0x7fe72c069b90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e970 tx=0x7fe72000ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe71e7fc700 1 -- 172.21.15.93:0/2948051824 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720009790 con 0x7fe72c0ff120 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe71e7fc700 1 -- 172.21.15.93:0/2948051824 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe72000eec0 con 0x7fe72c0ff120 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe72c10aa90 con 0x7fe72c0ff120 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe71e7fc700 1 -- 172.21.15.93:0/2948051824 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe720003cf0 con 0x7fe72c0ff120 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.683+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe72c10a900 con 0x7fe72c0ff120 2023-12-14T18:26:11.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.684+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe72c106610 con 0x7fe72c0ff120 2023-12-14T18:26:11.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.685+0000 7fe71e7fc700 1 -- 172.21.15.93:0/2948051824 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fe72000ccd0 con 0x7fe72c0ff120 2023-12-14T18:26:11.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.686+0000 7fe71e7fc700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe7180a11d0 0x7fe7180a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:11.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.686+0000 7fe7309fc700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe7180a11d0 0x7fe7180a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:11.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.686+0000 7fe71e7fc700 1 -- 172.21.15.93:0/2948051824 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fe720014070 con 0x7fe72c0ff120 2023-12-14T18:26:11.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.686+0000 7fe7309fc700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe7180a11d0 0x7fe7180a3680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe728005950 tx=0x7fe72800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:11.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.689+0000 7fe71e7fc700 1 -- 172.21.15.93:0/2948051824 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe7200963d0 con 0x7fe72c0ff120 2023-12-14T18:26:11.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.876+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe72c0619b0 con 0x7fe72c0ff120 2023-12-14T18:26:11.878 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.876+0000 7fe71e7fc700 1 -- 172.21.15.93:0/2948051824 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe7200999f0 con 0x7fe72c0ff120 2023-12-14T18:26:11.879 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:26:11.881 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe7180a11d0 msgr2=0x7fe7180a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe7180a11d0 0x7fe7180a3680 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fe728005950 tx=0x7fe72800a300 comp rx=0 tx=0).stop 2023-12-14T18:26:11.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c0ff120 msgr2=0x7fe72c069b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:11.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c0ff120 0x7fe72c069b90 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e970 tx=0x7fe72000ec80 comp rx=0 tx=0).stop 2023-12-14T18:26:11.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 shutdown_connections 2023-12-14T18:26:11.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fe7180a11d0 0x7fe7180a3680 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe72c0ff120 0x7fe72c069b90 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe72c100ca0 0x7fe72c19d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 --2- 172.21.15.93:0/2948051824 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe72c100310 0x7fe72c19ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:11.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 >> 172.21.15.93:0/2948051824 conn(0x7fe72c0fa740 msgr2=0x7fe72c103750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:11.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 shutdown_connections 2023-12-14T18:26:11.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:11.879+0000 7fe733461700 1 -- 172.21.15.93:0/2948051824 wait complete. 2023-12-14T18:26:11.893 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:26:12.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 -- 172.21.15.93:0/970586916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 msgr2=0x7f17c0108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.020 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 --2- 172.21.15.93:0/970586916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 0x7f17c0108350 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f17b800b0a0 tx=0x7f17b800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 -- 172.21.15.93:0/970586916 shutdown_connections 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 --2- 172.21.15.93:0/970586916 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 0x7f17c0108350 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 --2- 172.21.15.93:0/970586916 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c0100cb0 0x7f17c0103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 --2- 172.21.15.93:0/970586916 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c01035d0 0x7f17c01059b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 -- 172.21.15.93:0/970586916 >> 172.21.15.93:0/970586916 conn(0x7f17c00fa750 msgr2=0x7f17c00fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 -- 172.21.15.93:0/970586916 shutdown_connections 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 -- 172.21.15.93:0/970586916 wait complete. 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 Processor -- start 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 -- start start 2023-12-14T18:26:12.021 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c0100cb0 0x7f17c0196050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c01035d0 0x7f17c0196590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 0x7f17c019d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c019dbd0 con 0x7f17c0105ef0 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c019dd10 con 0x7f17c0100cb0 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c741a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17c019dea0 con 0x7f17c01035d0 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.016+0000 7f17c59b7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 0x7f17c019d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c59b7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 0x7f17c019d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46456/0 (socket says 172.21.15.93:46456) 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c59b7700 1 -- 172.21.15.93:0/2395835171 learned_addr learned my addr 172.21.15.93:0/2395835171 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c49b5700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c01035d0 0x7f17c0196590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c51b6700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c0100cb0 0x7f17c0196050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.022 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c59b7700 1 -- 172.21.15.93:0/2395835171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c01035d0 msgr2=0x7f17c0196590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c59b7700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c01035d0 0x7f17c0196590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c59b7700 1 -- 172.21.15.93:0/2395835171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c0100cb0 msgr2=0x7f17c0196050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c59b7700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c0100cb0 0x7f17c0196050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c59b7700 1 -- 172.21.15.93:0/2395835171 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17b8009d00 con 0x7f17c0105ef0 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c51b6700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c0100cb0 0x7f17c0196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c49b5700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c01035d0 0x7f17c0196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c59b7700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 0x7f17c019d600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f17b80062a0 tx=0x7f17b8004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17b67fc700 1 -- 172.21.15.93:0/2395835171 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b800e040 con 0x7f17c0105ef0 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17b67fc700 1 -- 172.21.15.93:0/2395835171 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f17b801db50 con 0x7f17c0105ef0 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17c010bbf0 con 0x7f17c0105ef0 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17b67fc700 1 -- 172.21.15.93:0/2395835171 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17b801bc40 con 0x7f17c0105ef0 2023-12-14T18:26:12.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.017+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17c010ba60 con 0x7f17c0105ef0 2023-12-14T18:26:12.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.018+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17c00fc2c0 con 0x7f17c0105ef0 2023-12-14T18:26:12.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.019+0000 7f17b67fc700 1 -- 172.21.15.93:0/2395835171 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f17b801d690 con 0x7f17c0105ef0 2023-12-14T18:26:12.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.020+0000 7f17b67fc700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f17ac0a11d0 0x7f17ac0a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.020+0000 7f17b67fc700 1 -- 172.21.15.93:0/2395835171 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f17b80ceef0 con 0x7f17c0105ef0 2023-12-14T18:26:12.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.020+0000 7f17c51b6700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f17ac0a11d0 0x7f17ac0a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.020+0000 7f17c51b6700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f17ac0a11d0 0x7f17ac0a3680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f17bc005fd0 tx=0x7f17bc005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:12.025 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.023+0000 7f17b67fc700 1 -- 172.21.15.93:0/2395835171 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f17b80983c0 con 0x7f17c0105ef0 2023-12-14T18:26:12.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.215+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f17c00619b0 con 0x7f17c0105ef0 2023-12-14T18:26:12.217 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.216+0000 7f17b67fc700 1 -- 172.21.15.93:0/2395835171 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f17b809b9e0 con 0x7f17c0105ef0 2023-12-14T18:26:12.218 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:26:12.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f17ac0a11d0 msgr2=0x7f17ac0a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f17ac0a11d0 0x7f17ac0a3680 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f17bc005fd0 tx=0x7f17bc005ee0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 msgr2=0x7f17c019d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 0x7f17c019d600 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f17b80062a0 tx=0x7f17b8004db0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 shutdown_connections 2023-12-14T18:26:12.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f17ac0a11d0 0x7f17ac0a3680 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f17c0105ef0 0x7f17c019d600 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f17c0100cb0 0x7f17c0196050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 --2- 172.21.15.93:0/2395835171 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f17c01035d0 0x7f17c0196590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 >> 172.21.15.93:0/2395835171 conn(0x7f17c00fa750 msgr2=0x7f17c0101910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:12.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 shutdown_connections 2023-12-14T18:26:12.221 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.218+0000 7f17c741a700 1 -- 172.21.15.93:0/2395835171 wait complete. 2023-12-14T18:26:12.231 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:12 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3237520275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:12.231 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:12 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2948051824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:12.232 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:26:12.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.355+0000 7f5ed1613700 1 -- 172.21.15.93:0/1649616057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 msgr2=0x7f5ecc100be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.355+0000 7f5ed1613700 1 --2- 172.21.15.93:0/1649616057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 0x7f5ecc100be0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0009990 tx=0x7f5ec0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.355+0000 7f5ed1613700 1 -- 172.21.15.93:0/1649616057 shutdown_connections 2023-12-14T18:26:12.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.355+0000 7f5ed1613700 1 --2- 172.21.15.93:0/1649616057 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 0x7f5ecc100be0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.355+0000 7f5ed1613700 1 --2- 172.21.15.93:0/1649616057 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ffe20 0x7f5ecc1076d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.355+0000 7f5ed1613700 1 --2- 172.21.15.93:0/1649616057 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ecc0ff470 0x7f5ecc0ff8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.355+0000 7f5ed1613700 1 -- 172.21.15.93:0/1649616057 >> 172.21.15.93:0/1649616057 conn(0x7f5ecc0f9f70 msgr2=0x7f5ecc0fc3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.355+0000 7f5ed1613700 1 -- 172.21.15.93:0/1649616057 shutdown_connections 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.355+0000 7f5ed1613700 1 -- 172.21.15.93:0/1649616057 wait complete. 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ed1613700 1 Processor -- start 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ed1613700 1 -- start start 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ed1613700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff470 0x7f5ecc195ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ed1613700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ecc0ffe20 0x7f5ecc196530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ed1613700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 0x7f5ecc19d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ed1613700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc19db70 con 0x7f5ecc1007d0 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ed1613700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc19dcb0 con 0x7f5ecc0ffe20 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ed1613700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ecc19de40 con 0x7f5ecc0ff470 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ecb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 0x7f5ecc19d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ecb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 0x7f5ecc19d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46474/0 (socket says 172.21.15.93:46474) 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ecb7fe700 1 -- 172.21.15.93:0/2058036763 learned_addr learned my addr 172.21.15.93:0/2058036763 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ecaffd700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff470 0x7f5ecc195ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5eca7fc700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ecc0ffe20 0x7f5ecc196530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ecb7fe700 1 -- 172.21.15.93:0/2058036763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff470 msgr2=0x7f5ecc195ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ecb7fe700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff470 0x7f5ecc195ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ecb7fe700 1 -- 172.21.15.93:0/2058036763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ecc0ffe20 msgr2=0x7f5ecc196530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ecb7fe700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ecc0ffe20 0x7f5ecc196530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5ecb7fe700 1 -- 172.21.15.93:0/2058036763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec00095f0 con 0x7f5ecc1007d0 2023-12-14T18:26:12.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.356+0000 7f5eca7fc700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ecc0ffe20 0x7f5ecc196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:12.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.357+0000 7f5ecb7fe700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 0x7f5ecc19d5a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4007ee0 tx=0x7f5ec400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:12.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.357+0000 7f5eb3fff700 1 -- 172.21.15.93:0/2058036763 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4017070 con 0x7f5ecc1007d0 2023-12-14T18:26:12.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.357+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ecc10ab50 con 0x7f5ecc1007d0 2023-12-14T18:26:12.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.357+0000 7f5eb3fff700 1 -- 172.21.15.93:0/2058036763 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5ec400f040 con 0x7f5ecc1007d0 2023-12-14T18:26:12.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.357+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ecc10a9c0 con 0x7f5ecc1007d0 2023-12-14T18:26:12.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.357+0000 7f5eb3fff700 1 -- 172.21.15.93:0/2058036763 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40056e0 con 0x7f5ecc1007d0 2023-12-14T18:26:12.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.358+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ecc19e530 con 0x7f5ecc1007d0 2023-12-14T18:26:12.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.359+0000 7f5eb3fff700 1 -- 172.21.15.93:0/2058036763 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5ec400dbd0 con 0x7f5ecc1007d0 2023-12-14T18:26:12.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.359+0000 7f5eb3fff700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5eb40a1150 0x7f5eb40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.359+0000 7f5eb3fff700 1 -- 172.21.15.93:0/2058036763 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5ec40ccb90 con 0x7f5ecc1007d0 2023-12-14T18:26:12.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.359+0000 7f5ecaffd700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5eb40a1150 0x7f5eb40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.360+0000 7f5ecaffd700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5eb40a1150 0x7f5eb40a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0006010 tx=0x7f5ec000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:12.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.363+0000 7f5eb3fff700 1 -- 172.21.15.93:0/2058036763 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5ec4095fd0 con 0x7f5ecc1007d0 2023-12-14T18:26:12.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:12 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3237520275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:12.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:12 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2948051824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:12.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.549+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5ecc19eae0 con 0x7f5ecc1007d0 2023-12-14T18:26:12.552 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.550+0000 7f5eb3fff700 1 -- 172.21.15.93:0/2058036763 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5ec40995f0 con 0x7f5ecc1007d0 2023-12-14T18:26:12.552 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-12-14T18:26:12.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5eb40a1150 msgr2=0x7f5eb40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5eb40a1150 0x7f5eb40a3600 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f5ec0006010 tx=0x7f5ec000b540 comp rx=0 tx=0).stop 2023-12-14T18:26:12.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 msgr2=0x7f5ecc19d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.554 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 0x7f5ecc19d5a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7f5ec4007ee0 tx=0x7f5ec400d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 shutdown_connections 2023-12-14T18:26:12.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5eb40a1150 0x7f5eb40a3600 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5ecc1007d0 0x7f5ecc19d5a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5ecc0ff470 0x7f5ecc195ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 --2- 172.21.15.93:0/2058036763 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5ecc0ffe20 0x7f5ecc196530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 >> 172.21.15.93:0/2058036763 conn(0x7f5ecc0f9f70 msgr2=0x7f5ecc0fe380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:12.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 shutdown_connections 2023-12-14T18:26:12.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.552+0000 7f5ed1613700 1 -- 172.21.15.93:0/2058036763 wait complete. 2023-12-14T18:26:12.566 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:26:12.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:12 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3237520275' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:12.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:12 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2948051824' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:12.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 -- 172.21.15.93:0/976879011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100310 msgr2=0x7f5b68100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/976879011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100310 0x7f5b68100760 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f5b64009990 tx=0x7f5b64009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 -- 172.21.15.93:0/976879011 shutdown_connections 2023-12-14T18:26:12.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/976879011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b68100ca0 0x7f5b68107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/976879011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b680ff120 0x7f5b680ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/976879011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100310 0x7f5b68100760 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 -- 172.21.15.93:0/976879011 >> 172.21.15.93:0/976879011 conn(0x7f5b680fa740 msgr2=0x7f5b680fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:12.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 -- 172.21.15.93:0/976879011 shutdown_connections 2023-12-14T18:26:12.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 -- 172.21.15.93:0/976879011 wait complete. 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.692+0000 7f5b6fda0700 1 Processor -- start 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6fda0700 1 -- start start 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6fda0700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b680ff120 0x7f5b681030b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6fda0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b68100310 0x7f5b681016f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6fda0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100ca0 0x7f5b68101c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6fda0700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681023d0 con 0x7f5b68100ca0 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6fda0700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b68102540 con 0x7f5b68100310 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6fda0700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b681a0020 con 0x7f5b680ff120 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6e33d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100ca0 0x7f5b68101c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6e33d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100ca0 0x7f5b68101c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46500/0 (socket says 172.21.15.93:46500) 2023-12-14T18:26:12.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6e33d700 1 -- 172.21.15.93:0/636014195 learned_addr learned my addr 172.21.15.93:0/636014195 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6d33b700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b68100310 0x7f5b681016f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6db3c700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b680ff120 0x7f5b681030b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6e33d700 1 -- 172.21.15.93:0/636014195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b680ff120 msgr2=0x7f5b681030b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6e33d700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b680ff120 0x7f5b681030b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6e33d700 1 -- 172.21.15.93:0/636014195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b68100310 msgr2=0x7f5b681016f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6e33d700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b68100310 0x7f5b681016f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6e33d700 1 -- 172.21.15.93:0/636014195 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b640095f0 con 0x7f5b68100ca0 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6db3c700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b680ff120 0x7f5b681030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6d33b700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b68100310 0x7f5b681016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6e33d700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100ca0 0x7f5b68101c30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5b60007ee0 tx=0x7f5b6000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b5effd700 1 -- 172.21.15.93:0/636014195 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000ddd0 con 0x7f5b68100ca0 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b5effd700 1 -- 172.21.15.93:0/636014195 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5b6000f040 con 0x7f5b68100ca0 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b5effd700 1 -- 172.21.15.93:0/636014195 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b60015400 con 0x7f5b68100ca0 2023-12-14T18:26:12.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.693+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b6810aa80 con 0x7f5b68100ca0 2023-12-14T18:26:12.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.694+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6810a8f0 con 0x7f5b68100ca0 2023-12-14T18:26:12.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.694+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b681a0710 con 0x7f5b68100ca0 2023-12-14T18:26:12.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.695+0000 7f5b5effd700 1 -- 172.21.15.93:0/636014195 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f5b60004020 con 0x7f5b68100ca0 2023-12-14T18:26:12.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.696+0000 7f5b5effd700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5b540a11d0 0x7f5b540a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:12.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.696+0000 7f5b5effd700 1 -- 172.21.15.93:0/636014195 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f5b600cc500 con 0x7f5b68100ca0 2023-12-14T18:26:12.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.696+0000 7f5b6db3c700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5b540a11d0 0x7f5b540a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:12.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.696+0000 7f5b6db3c700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5b540a11d0 0x7f5b540a3680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5b58005fd0 tx=0x7f5b58005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:12.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.700+0000 7f5b5effd700 1 -- 172.21.15.93:0/636014195 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f5b6004e730 con 0x7f5b68100ca0 2023-12-14T18:26:12.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.889+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f5b681a0cc0 con 0x7f5b68100ca0 2023-12-14T18:26:12.891 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.890+0000 7f5b5effd700 1 -- 172.21.15.93:0/636014195 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7f5b60095940 con 0x7f5b68100ca0 2023-12-14T18:26:12.892 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:26:12.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5b540a11d0 msgr2=0x7f5b540a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5b540a11d0 0x7f5b540a3680 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5b58005fd0 tx=0x7f5b58005dc0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.894 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100ca0 msgr2=0x7f5b68101c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:12.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100ca0 0x7f5b68101c30 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f5b60007ee0 tx=0x7f5b6000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 shutdown_connections 2023-12-14T18:26:12.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f5b540a11d0 0x7f5b540a3680 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5b68100ca0 0x7f5b68101c30 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5b680ff120 0x7f5b681030b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 --2- 172.21.15.93:0/636014195 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5b68100310 0x7f5b681016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:12.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 >> 172.21.15.93:0/636014195 conn(0x7f5b680fa740 msgr2=0x7f5b68067df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:12.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 shutdown_connections 2023-12-14T18:26:12.895 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:12.892+0000 7f5b6fda0700 1 -- 172.21.15.93:0/636014195 wait complete. 2023-12-14T18:26:12.906 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:26:13.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.026+0000 7f47b58cb700 1 -- 172.21.15.93:0/1454322409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b0101570 msgr2=0x7f47b01019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.026+0000 7f47b58cb700 1 --2- 172.21.15.93:0/1454322409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b0101570 0x7f47b01019c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f4798009990 tx=0x7f4798009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.026+0000 7f47b58cb700 1 -- 172.21.15.93:0/1454322409 shutdown_connections 2023-12-14T18:26:13.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.026+0000 7f47b58cb700 1 --2- 172.21.15.93:0/1454322409 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b0101f00 0x7f47b01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.026+0000 7f47b58cb700 1 --2- 172.21.15.93:0/1454322409 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b00fe5a0 0x7f47b00fe9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.026+0000 7f47b58cb700 1 --2- 172.21.15.93:0/1454322409 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b0101570 0x7f47b01019c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.026+0000 7f47b58cb700 1 -- 172.21.15.93:0/1454322409 >> 172.21.15.93:0/1454322409 conn(0x7f47b00f9f80 msgr2=0x7f47b00fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.026+0000 7f47b58cb700 1 -- 172.21.15.93:0/1454322409 shutdown_connections 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.026+0000 7f47b58cb700 1 -- 172.21.15.93:0/1454322409 wait complete. 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47b58cb700 1 Processor -- start 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47b58cb700 1 -- start start 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47b58cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b00fe5a0 0x7f47b0100ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47b58cb700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b0101570 0x7f47b00ff520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47b58cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0101f00 0x7f47b00ffa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47b58cb700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0100200 con 0x7f47b00fe5a0 2023-12-14T18:26:13.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47b58cb700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b0100370 con 0x7f47b0101f00 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47b58cb700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47b019ff70 con 0x7f47b0101570 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47aeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b00fe5a0 0x7f47b0100ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47aeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b00fe5a0 0x7f47b0100ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46510/0 (socket says 172.21.15.93:46510) 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47aeffd700 1 -- 172.21.15.93:0/2397114336 learned_addr learned my addr 172.21.15.93:0/2397114336 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47ae7fc700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b0101570 0x7f47b00ff520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47af7fe700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0101f00 0x7f47b00ffa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47aeffd700 1 -- 172.21.15.93:0/2397114336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b0101570 msgr2=0x7f47b00ff520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47aeffd700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b0101570 0x7f47b00ff520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47aeffd700 1 -- 172.21.15.93:0/2397114336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0101f00 msgr2=0x7f47b00ffa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47aeffd700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0101f00 0x7f47b00ffa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47aeffd700 1 -- 172.21.15.93:0/2397114336 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47980095f0 con 0x7f47b00fe5a0 2023-12-14T18:26:13.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.027+0000 7f47af7fe700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0101f00 0x7f47b00ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:13.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.028+0000 7f47aeffd700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b00fe5a0 0x7f47b0100ee0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f47a000c990 tx=0x7f47a000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:13.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.028+0000 7f47b48c9700 1 -- 172.21.15.93:0/2397114336 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0007b00 con 0x7f47b00fe5a0 2023-12-14T18:26:13.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.028+0000 7f47b48c9700 1 -- 172.21.15.93:0/2397114336 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f47a0007c60 con 0x7f47b00fe5a0 2023-12-14T18:26:13.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.028+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47b010bbd0 con 0x7f47b00fe5a0 2023-12-14T18:26:13.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.028+0000 7f47b48c9700 1 -- 172.21.15.93:0/2397114336 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47a0018660 con 0x7f47b00fe5a0 2023-12-14T18:26:13.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.028+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47b010ba40 con 0x7f47b00fe5a0 2023-12-14T18:26:13.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.029+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47b0107890 con 0x7f47b00fe5a0 2023-12-14T18:26:13.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.030+0000 7f47b48c9700 1 -- 172.21.15.93:0/2397114336 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f47a000f450 con 0x7f47b00fe5a0 2023-12-14T18:26:13.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.031+0000 7f47b48c9700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f479c0a1150 0x7f479c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.031+0000 7f47ae7fc700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f479c0a1150 0x7f479c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.031+0000 7f47b48c9700 1 -- 172.21.15.93:0/2397114336 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f47a00ccc40 con 0x7f47b00fe5a0 2023-12-14T18:26:13.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.032+0000 7f47ae7fc700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f479c0a1150 0x7f479c0a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f479800b5c0 tx=0x7f4798005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:13.036 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.034+0000 7f47b48c9700 1 -- 172.21.15.93:0/2397114336 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f47a0096110 con 0x7f47b00fe5a0 2023-12-14T18:26:13.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.269+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f47b002cfe0 con 0x7f47b00fe5a0 2023-12-14T18:26:13.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.269+0000 7f47b48c9700 1 -- 172.21.15.93:0/2397114336 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10799 (secure 0 0 0) 0x7f47a0099730 con 0x7f47b00fe5a0 2023-12-14T18:26:13.271 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:26:13.272 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:25:01.760987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:26:13.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f479c0a1150 msgr2=0x7f479c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f479c0a1150 0x7f479c0a3600 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f479800b5c0 tx=0x7f4798005fd0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b00fe5a0 msgr2=0x7f47b0100ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b00fe5a0 0x7f47b0100ee0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f47a000c990 tx=0x7f47a000cca0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.274 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 shutdown_connections 2023-12-14T18:26:13.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f479c0a1150 0x7f479c0a3600 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f47b00fe5a0 0x7f47b0100ee0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f47b0101f00 0x7f47b00ffa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 --2- 172.21.15.93:0/2397114336 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f47b0101570 0x7f47b00ff520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.271+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 >> 172.21.15.93:0/2397114336 conn(0x7f47b00f9f80 msgr2=0x7f47b01049d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:13.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.272+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 shutdown_connections 2023-12-14T18:26:13.275 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.272+0000 7f47b58cb700 1 -- 172.21.15.93:0/2397114336 wait complete. 2023-12-14T18:26:13.275 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 24 2023-12-14T18:26:13.285 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:13 smithi093 ceph-mon[42070]: pgmap v274: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:26:13.285 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:13 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2395835171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:13.285 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:13 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2058036763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:13.285 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:13 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/636014195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:13.287 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.407+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/247227156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 msgr2=0x7ffa58101cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.407+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/247227156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 0x7ffa58101cd0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7ffa48009990 tx=0x7ffa48009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.407+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/247227156 shutdown_connections 2023-12-14T18:26:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.407+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/247227156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 0x7ffa58101cd0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.407+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/247227156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa580fecc0 0x7ffa581088a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.407+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/247227156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa580fe330 0x7ffa580fe780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.407+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/247227156 >> 172.21.15.93:0/247227156 conn(0x7ffa580f9f80 msgr2=0x7ffa580fc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.407+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/247227156 shutdown_connections 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.407+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/247227156 wait complete. 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.408+0000 7ffa5d5d2700 1 Processor -- start 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.408+0000 7ffa5d5d2700 1 -- start start 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.408+0000 7ffa5d5d2700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa580fe330 0x7ffa58195fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.408+0000 7ffa5d5d2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa580fecc0 0x7ffa58196500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.408+0000 7ffa5d5d2700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 0x7ffa5819d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.408+0000 7ffa5d5d2700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5819db40 con 0x7ffa581018c0 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.408+0000 7ffa5d5d2700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5819dc80 con 0x7ffa580fecc0 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.408+0000 7ffa5d5d2700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa5819de10 con 0x7ffa580fe330 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.408+0000 7ffa577fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 0x7ffa5819d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa577fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 0x7ffa5819d570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46516/0 (socket says 172.21.15.93:46516) 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa577fe700 1 -- 172.21.15.93:0/3972528912 learned_addr learned my addr 172.21.15.93:0/3972528912 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa56ffd700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa580fe330 0x7ffa58195fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa567fc700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa580fecc0 0x7ffa58196500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa577fe700 1 -- 172.21.15.93:0/3972528912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa580fe330 msgr2=0x7ffa58195fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa577fe700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa580fe330 0x7ffa58195fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa577fe700 1 -- 172.21.15.93:0/3972528912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa580fecc0 msgr2=0x7ffa58196500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa577fe700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa580fecc0 0x7ffa58196500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa577fe700 1 -- 172.21.15.93:0/3972528912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa480095f0 con 0x7ffa581018c0 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa567fc700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa580fecc0 0x7ffa58196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa577fe700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 0x7ffa5819d570 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa50007ee0 tx=0x7ffa5000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa3ffff700 1 -- 172.21.15.93:0/3972528912 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50017070 con 0x7ffa581018c0 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa5810bbc0 con 0x7ffa581018c0 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa3ffff700 1 -- 172.21.15.93:0/3972528912 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ffa5000f040 con 0x7ffa581018c0 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa3ffff700 1 -- 172.21.15.93:0/3972528912 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa50005740 con 0x7ffa581018c0 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.409+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa5810ba30 con 0x7ffa581018c0 2023-12-14T18:26:13.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.410+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa581079f0 con 0x7ffa581018c0 2023-12-14T18:26:13.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.411+0000 7ffa3ffff700 1 -- 172.21.15.93:0/3972528912 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7ffa5000dc60 con 0x7ffa581018c0 2023-12-14T18:26:13.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.412+0000 7ffa3ffff700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffa400a11d0 0x7ffa400a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.412+0000 7ffa56ffd700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffa400a11d0 0x7ffa400a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.412+0000 7ffa3ffff700 1 -- 172.21.15.93:0/3972528912 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7ffa500ccad0 con 0x7ffa581018c0 2023-12-14T18:26:13.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.412+0000 7ffa56ffd700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffa400a11d0 0x7ffa400a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800b5c0 tx=0x7ffa48005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:13.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.415+0000 7ffa3ffff700 1 -- 172.21.15.93:0/3972528912 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7ffa50095eb0 con 0x7ffa581018c0 2023-12-14T18:26:13.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:13 smithi160 ceph-mon[43840]: pgmap v274: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:26:13.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:13 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2395835171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:13.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:13 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2058036763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:13.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:13 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/636014195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:13.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:13 smithi097 ceph-mon[43486]: pgmap v274: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:26:13.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:13 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2395835171' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:13.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:13 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2058036763' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:13.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:13 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/636014195' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:13.606 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.604+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ffa5802cfe0 con 0x7ffa581018c0 2023-12-14T18:26:13.608 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.606+0000 7ffa3ffff700 1 -- 172.21.15.93:0/3972528912 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24357 (secure 0 0 0) 0x7ffa500994d0 con 0x7ffa581018c0 2023-12-14T18:26:13.608 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:26:13.608 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":86,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:25:37.836200+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:13.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffa400a11d0 msgr2=0x7ffa400a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffa400a11d0 0x7ffa400a3680 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7ffa4800b5c0 tx=0x7ffa48005fd0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.610 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 msgr2=0x7ffa5819d570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 0x7ffa5819d570 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7ffa50007ee0 tx=0x7ffa5000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 shutdown_connections 2023-12-14T18:26:13.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7ffa400a11d0 0x7ffa400a3680 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ffa581018c0 0x7ffa5819d570 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ffa580fe330 0x7ffa58195fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 --2- 172.21.15.93:0/3972528912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ffa580fecc0 0x7ffa58196500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 >> 172.21.15.93:0/3972528912 conn(0x7ffa580f9f80 msgr2=0x7ffa58104b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:13.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 shutdown_connections 2023-12-14T18:26:13.611 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.608+0000 7ffa5d5d2700 1 -- 172.21.15.93:0/3972528912 wait complete. 2023-12-14T18:26:13.623 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:24:34.407045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:24:35.728548+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '80', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:24:38.796189+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:26:13.623 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:26:13.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.746+0000 7f57d2aae700 1 -- 172.21.15.93:0/433900106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc100320 msgr2=0x7f57cc100770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 --2- 172.21.15.93:0/433900106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc100320 0x7f57cc100770 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f57b8009990 tx=0x7f57b8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.751 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 -- 172.21.15.93:0/433900106 shutdown_connections 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 --2- 172.21.15.93:0/433900106 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57cc100cb0 0x7f57cc107570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 --2- 172.21.15.93:0/433900106 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57cc0ff130 0x7f57cc0ff540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 --2- 172.21.15.93:0/433900106 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc100320 0x7f57cc100770 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 -- 172.21.15.93:0/433900106 >> 172.21.15.93:0/433900106 conn(0x7f57cc0fa750 msgr2=0x7f57cc0fcb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 -- 172.21.15.93:0/433900106 shutdown_connections 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 -- 172.21.15.93:0/433900106 wait complete. 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 Processor -- start 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 -- start start 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc0ff130 0x7f57cc195f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57cc100320 0x7f57cc1964c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57cc100cb0 0x7f57cc19d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19db00 con 0x7f57cc0ff130 2023-12-14T18:26:13.752 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19dc40 con 0x7f57cc100cb0 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.747+0000 7f57d2aae700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57cc19ddd0 con 0x7f57cc100320 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d084a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc0ff130 0x7f57cc195f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d084a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc0ff130 0x7f57cc195f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46526/0 (socket says 172.21.15.93:46526) 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d084a700 1 -- 172.21.15.93:0/1752592672 learned_addr learned my addr 172.21.15.93:0/1752592672 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57cbfff700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57cc100320 0x7f57cc1964c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d104b700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57cc100cb0 0x7f57cc19d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d084a700 1 -- 172.21.15.93:0/1752592672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57cc100320 msgr2=0x7f57cc1964c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d084a700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57cc100320 0x7f57cc1964c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d084a700 1 -- 172.21.15.93:0/1752592672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57cc100cb0 msgr2=0x7f57cc19d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d084a700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57cc100cb0 0x7f57cc19d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d084a700 1 -- 172.21.15.93:0/1752592672 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57b80095f0 con 0x7f57cc0ff130 2023-12-14T18:26:13.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d104b700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57cc100cb0 0x7f57cc19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:13.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d084a700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc0ff130 0x7f57cc195f80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f57c000d980 tx=0x7f57c000dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:13.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57c9ffb700 1 -- 172.21.15.93:0/1752592672 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c000de60 con 0x7f57cc0ff130 2023-12-14T18:26:13.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57c9ffb700 1 -- 172.21.15.93:0/1752592672 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f57c00044e0 con 0x7f57cc0ff130 2023-12-14T18:26:13.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57cc10aa90 con 0x7f57cc0ff130 2023-12-14T18:26:13.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57c9ffb700 1 -- 172.21.15.93:0/1752592672 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57c0003be0 con 0x7f57cc0ff130 2023-12-14T18:26:13.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.748+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57cc10a900 con 0x7f57cc0ff130 2023-12-14T18:26:13.754 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.749+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f57cc107260 con 0x7f57cc0ff130 2023-12-14T18:26:13.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.750+0000 7f57c9ffb700 1 -- 172.21.15.93:0/1752592672 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f57c0009bb0 con 0x7f57cc0ff130 2023-12-14T18:26:13.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.751+0000 7f57c9ffb700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57bc0a1150 0x7f57bc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:13.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.751+0000 7f57cbfff700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57bc0a1150 0x7f57bc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:13.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.751+0000 7f57c9ffb700 1 -- 172.21.15.93:0/1752592672 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f57c00cc9d0 con 0x7f57cc0ff130 2023-12-14T18:26:13.755 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.751+0000 7f57cbfff700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57bc0a1150 0x7f57bc0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b5c0 tx=0x7f57b8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:13.756 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.755+0000 7f57c9ffb700 1 -- 172.21.15.93:0/1752592672 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f57c0095e10 con 0x7f57cc0ff130 2023-12-14T18:26:13.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.942+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f57cc061700 con 0x7f57cc0ff130 2023-12-14T18:26:13.944 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.942+0000 7f57c9ffb700 1 -- 172.21.15.93:0/1752592672 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v86) v1 ==== 93+0+10 (secure 0 0 0) 0x7f57c0009940 con 0x7f57cc0ff130 2023-12-14T18:26:13.944 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57bc0a1150 msgr2=0x7f57bc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57bc0a1150 0x7f57bc0a3600 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f57b800b5c0 tx=0x7f57b8005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc0ff130 msgr2=0x7f57cc195f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc0ff130 0x7f57cc195f80 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f57c000d980 tx=0x7f57c000dd40 comp rx=0 tx=0).stop 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 shutdown_connections 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f57bc0a1150 0x7f57bc0a3600 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57cc0ff130 0x7f57cc195f80 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57cc100cb0 0x7f57cc19d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 --2- 172.21.15.93:0/1752592672 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57cc100320 0x7f57cc1964c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 >> 172.21.15.93:0/1752592672 conn(0x7f57cc0fa750 msgr2=0x7f57cc0fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 shutdown_connections 2023-12-14T18:26:13.947 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:13.945+0000 7f57d2aae700 1 -- 172.21.15.93:0/1752592672 wait complete. 2023-12-14T18:26:13.959 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:26:14.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.081+0000 7f924f7fe700 1 -- 172.21.15.93:0/3744145042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248100230 msgr2=0x7f9248100640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.081+0000 7f924f7fe700 1 --2- 172.21.15.93:0/3744145042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248100230 0x7f9248100640 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f9238009990 tx=0x7f9238009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.081+0000 7f924f7fe700 1 -- 172.21.15.93:0/3744145042 shutdown_connections 2023-12-14T18:26:14.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.081+0000 7f924f7fe700 1 --2- 172.21.15.93:0/3744145042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248100230 0x7f9248100640 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.081+0000 7f924f7fe700 1 --2- 172.21.15.93:0/3744145042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9248101db0 0x7f9248108670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.083 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.081+0000 7f924f7fe700 1 --2- 172.21.15.93:0/3744145042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9248101420 0x7f9248101870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.081+0000 7f924f7fe700 1 -- 172.21.15.93:0/3744145042 >> 172.21.15.93:0/3744145042 conn(0x7f92480fb850 msgr2=0x7f92480fdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:14.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.081+0000 7f924f7fe700 1 -- 172.21.15.93:0/3744145042 shutdown_connections 2023-12-14T18:26:14.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.081+0000 7f924f7fe700 1 -- 172.21.15.93:0/3744145042 wait complete. 2023-12-14T18:26:14.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924f7fe700 1 Processor -- start 2023-12-14T18:26:14.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924f7fe700 1 -- start start 2023-12-14T18:26:14.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924f7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9248100230 0x7f9248195fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924f7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9248101420 0x7f9248196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248101db0 0x7f924819d580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924f7fe700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924819db50 con 0x7f9248101db0 2023-12-14T18:26:14.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924f7fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924819dc90 con 0x7f9248101420 2023-12-14T18:26:14.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924f7fe700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924819de20 con 0x7f9248100230 2023-12-14T18:26:14.086 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248101db0 0x7f924819d580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248101db0 0x7f924819d580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46536/0 (socket says 172.21.15.93:46536) 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.082+0000 7f924dd9b700 1 -- 172.21.15.93:0/1156484907 learned_addr learned my addr 172.21.15.93:0/1156484907 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924d59a700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9248100230 0x7f9248195fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924cd99700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9248101420 0x7f9248196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924dd9b700 1 -- 172.21.15.93:0/1156484907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9248100230 msgr2=0x7f9248195fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924dd9b700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9248100230 0x7f9248195fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924dd9b700 1 -- 172.21.15.93:0/1156484907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9248101420 msgr2=0x7f9248196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924dd9b700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9248101420 0x7f9248196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924dd9b700 1 -- 172.21.15.93:0/1156484907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92380095f0 con 0x7f9248101db0 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924cd99700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9248101420 0x7f9248196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924dd9b700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248101db0 0x7f924819d580 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9240007ee0 tx=0x7f924000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f923e7fc700 1 -- 172.21.15.93:0/1156484907 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9240017070 con 0x7f9248101db0 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f923e7fc700 1 -- 172.21.15.93:0/1156484907 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f924000f040 con 0x7f9248101db0 2023-12-14T18:26:14.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924810bba0 con 0x7f9248101db0 2023-12-14T18:26:14.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f923e7fc700 1 -- 172.21.15.93:0/1156484907 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9240005740 con 0x7f9248101db0 2023-12-14T18:26:14.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.083+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924810ba10 con 0x7f9248101db0 2023-12-14T18:26:14.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.084+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92481077c0 con 0x7f9248101db0 2023-12-14T18:26:14.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.085+0000 7f923e7fc700 1 -- 172.21.15.93:0/1156484907 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f924000dc60 con 0x7f9248101db0 2023-12-14T18:26:14.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.086+0000 7f923e7fc700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f92340a11d0 0x7f92340a3680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.086+0000 7f924d59a700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f92340a11d0 0x7f92340a3680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.086+0000 7f923e7fc700 1 -- 172.21.15.93:0/1156484907 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f92400cc430 con 0x7f9248101db0 2023-12-14T18:26:14.089 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.086+0000 7f924d59a700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f92340a11d0 0x7f92340a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9238006010 tx=0x7f9238005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:14.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.090+0000 7f923e7fc700 1 -- 172.21.15.93:0/1156484907 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9240095810 con 0x7f9248101db0 2023-12-14T18:26:14.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.277+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9248061930 con 0x7f9248101db0 2023-12-14T18:26:14.279 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.278+0000 7f923e7fc700 1 -- 172.21.15.93:0/1156484907 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v86) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9240098e30 con 0x7f9248101db0 2023-12-14T18:26:14.279 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f92340a11d0 msgr2=0x7f92340a3680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f92340a11d0 0x7f92340a3680 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f9238006010 tx=0x7f9238005960 comp rx=0 tx=0).stop 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248101db0 msgr2=0x7f924819d580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248101db0 0x7f924819d580 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f9240007ee0 tx=0x7f924000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 shutdown_connections 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f92340a11d0 0x7f92340a3680 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9248101db0 0x7f924819d580 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9248100230 0x7f9248195fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 --2- 172.21.15.93:0/1156484907 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9248101420 0x7f9248196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 >> 172.21.15.93:0/1156484907 conn(0x7f92480fb850 msgr2=0x7f9248104900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 shutdown_connections 2023-12-14T18:26:14.282 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.280+0000 7f924f7fe700 1 -- 172.21.15.93:0/1156484907 wait complete. 2023-12-14T18:26:14.293 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:14 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2397114336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:14.293 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:14 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3972528912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:14.293 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:14 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1752592672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:14.294 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:26:14.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.417+0000 7fce290fe700 1 -- 172.21.15.93:0/3886162121 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 msgr2=0x7fce240ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.417+0000 7fce290fe700 1 --2- 172.21.15.93:0/3886162121 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 0x7fce240ff7e0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fce14009990 tx=0x7fce14009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.419 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.417+0000 7fce290fe700 1 -- 172.21.15.93:0/3886162121 shutdown_connections 2023-12-14T18:26:14.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.417+0000 7fce290fe700 1 --2- 172.21.15.93:0/3886162121 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce240ffd20 0x7fce241075d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.417+0000 7fce290fe700 1 --2- 172.21.15.93:0/3886162121 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce24100720 0x7fce24100b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.417+0000 7fce290fe700 1 --2- 172.21.15.93:0/3886162121 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 0x7fce240ff7e0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.417+0000 7fce290fe700 1 -- 172.21.15.93:0/3886162121 >> 172.21.15.93:0/3886162121 conn(0x7fce240f9ed0 msgr2=0x7fce240fc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:14.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.417+0000 7fce290fe700 1 -- 172.21.15.93:0/3886162121 shutdown_connections 2023-12-14T18:26:14.420 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.417+0000 7fce290fe700 1 -- 172.21.15.93:0/3886162121 wait complete. 2023-12-14T18:26:14.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce290fe700 1 Processor -- start 2023-12-14T18:26:14.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce290fe700 1 -- start start 2023-12-14T18:26:14.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce290fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 0x7fce240fedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce290fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce240ffd20 0x7fce2419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.422 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce290fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce24100720 0x7fce2419d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce290fe700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419daf0 con 0x7fce240ff370 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce290fe700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419dc60 con 0x7fce240ffd20 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce290fe700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2419de20 con 0x7fce24100720 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce22d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 0x7fce240fedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce22d9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 0x7fce240fedf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46550/0 (socket says 172.21.15.93:46550) 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.418+0000 7fce22d9d700 1 -- 172.21.15.93:0/169325209 learned_addr learned my addr 172.21.15.93:0/169325209 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce2359e700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce24100720 0x7fce2419d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce2259c700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce240ffd20 0x7fce2419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce22d9d700 1 -- 172.21.15.93:0/169325209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce24100720 msgr2=0x7fce2419d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce22d9d700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce24100720 0x7fce2419d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce22d9d700 1 -- 172.21.15.93:0/169325209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce240ffd20 msgr2=0x7fce2419acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce22d9d700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce240ffd20 0x7fce2419acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce22d9d700 1 -- 172.21.15.93:0/169325209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce140095f0 con 0x7fce240ff370 2023-12-14T18:26:14.423 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce2259c700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce240ffd20 0x7fce2419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:14.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce22d9d700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 0x7fce240fedf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00d980 tx=0x7fce0c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:14.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce1bfff700 1 -- 172.21.15.93:0/169325209 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c009b40 con 0x7fce240ff370 2023-12-14T18:26:14.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2410aaf0 con 0x7fce240ff370 2023-12-14T18:26:14.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce1bfff700 1 -- 172.21.15.93:0/169325209 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce0c00de60 con 0x7fce240ff370 2023-12-14T18:26:14.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce1bfff700 1 -- 172.21.15.93:0/169325209 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0c00f620 con 0x7fce240ff370 2023-12-14T18:26:14.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.419+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2410a960 con 0x7fce240ff370 2023-12-14T18:26:14.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.420+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce24106110 con 0x7fce240ff370 2023-12-14T18:26:14.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.422+0000 7fce1bfff700 1 -- 172.21.15.93:0/169325209 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fce0c010460 con 0x7fce240ff370 2023-12-14T18:26:14.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.422+0000 7fce1bfff700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fce100a1150 0x7fce100a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.422+0000 7fce1bfff700 1 -- 172.21.15.93:0/169325209 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fce0c0cc840 con 0x7fce240ff370 2023-12-14T18:26:14.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.422+0000 7fce2259c700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fce100a1150 0x7fce100a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.425 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.423+0000 7fce2259c700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fce100a1150 0x7fce100a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fce1400b5c0 tx=0x7fce14005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:14.427 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.425+0000 7fce1bfff700 1 -- 172.21.15.93:0/169325209 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fce0c095c80 con 0x7fce240ff370 2023-12-14T18:26:14.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:14 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2397114336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:14.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:14 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3972528912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:14.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:14 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1752592672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:14.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:14 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2397114336' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:14.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:14 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3972528912' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:14.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:14 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1752592672' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:26:14.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.614+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fce24061b20 con 0x7fce240ff370 2023-12-14T18:26:14.616 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.615+0000 7fce1bfff700 1 -- 172.21.15.93:0/169325209 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v86) v1 ==== 100+0+11 (secure 0 0 0) 0x7fce0c017020 con 0x7fce240ff370 2023-12-14T18:26:14.617 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-12-14T18:26:14.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fce100a1150 msgr2=0x7fce100a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fce100a1150 0x7fce100a3600 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fce1400b5c0 tx=0x7fce14005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 msgr2=0x7fce240fedf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 0x7fce240fedf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fce0c00d980 tx=0x7fce0c00dd40 comp rx=0 tx=0).stop 2023-12-14T18:26:14.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 shutdown_connections 2023-12-14T18:26:14.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fce100a1150 0x7fce100a3600 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.619 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce240ff370 0x7fce240fedf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce24100720 0x7fce2419d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 --2- 172.21.15.93:0/169325209 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce240ffd20 0x7fce2419acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 >> 172.21.15.93:0/169325209 conn(0x7fce240f9ed0 msgr2=0x7fce240fc170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:14.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 shutdown_connections 2023-12-14T18:26:14.620 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.617+0000 7fce290fe700 1 -- 172.21.15.93:0/169325209 wait complete. 2023-12-14T18:26:14.631 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:26:14.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.755+0000 7fa0df74a700 1 -- 172.21.15.93:0/2397026709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100310 msgr2=0x7fa0d8100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.755+0000 7fa0df74a700 1 --2- 172.21.15.93:0/2397026709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100310 0x7fa0d8100760 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fa0d4009990 tx=0x7fa0d4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.755+0000 7fa0df74a700 1 -- 172.21.15.93:0/2397026709 shutdown_connections 2023-12-14T18:26:14.759 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.755+0000 7fa0df74a700 1 --2- 172.21.15.93:0/2397026709 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0d8100ca0 0x7fa0d8107560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.755+0000 7fa0df74a700 1 --2- 172.21.15.93:0/2397026709 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d80ff120 0x7fa0d80ff530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.755+0000 7fa0df74a700 1 --2- 172.21.15.93:0/2397026709 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100310 0x7fa0d8100760 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.755+0000 7fa0df74a700 1 -- 172.21.15.93:0/2397026709 >> 172.21.15.93:0/2397026709 conn(0x7fa0d80fa740 msgr2=0x7fa0d80fcb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.755+0000 7fa0df74a700 1 -- 172.21.15.93:0/2397026709 shutdown_connections 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.755+0000 7fa0df74a700 1 -- 172.21.15.93:0/2397026709 wait complete. 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0df74a700 1 Processor -- start 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0df74a700 1 -- start start 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0df74a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d80ff120 0x7fa0d8194e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0df74a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0d8100310 0x7fa0d8195360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0df74a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100ca0 0x7fa0d819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0df74a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d819c9a0 con 0x7fa0d8100ca0 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0df74a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d819cae0 con 0x7fa0d80ff120 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0df74a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa0d819cc70 con 0x7fa0d8100310 2023-12-14T18:26:14.760 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ddce7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100ca0 0x7fa0d819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ddce7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100ca0 0x7fa0d819c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46576/0 (socket says 172.21.15.93:46576) 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ddce7700 1 -- 172.21.15.93:0/3070827146 learned_addr learned my addr 172.21.15.93:0/3070827146 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0dcce5700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0d8100310 0x7fa0d8195360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0dd4e6700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d80ff120 0x7fa0d8194e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ddce7700 1 -- 172.21.15.93:0/3070827146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0d8100310 msgr2=0x7fa0d8195360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ddce7700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0d8100310 0x7fa0d8195360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ddce7700 1 -- 172.21.15.93:0/3070827146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d80ff120 msgr2=0x7fa0d8194e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ddce7700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d80ff120 0x7fa0d8194e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ddce7700 1 -- 172.21.15.93:0/3070827146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0d40095f0 con 0x7fa0d8100ca0 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0dcce5700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0d8100310 0x7fa0d8195360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ddce7700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100ca0 0x7fa0d819c3d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0007f10 tx=0x7fa0d000f5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.756+0000 7fa0ce7fc700 1 -- 172.21.15.93:0/3070827146 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d000fea0 con 0x7fa0d8100ca0 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.757+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0d810aa80 con 0x7fa0d8100ca0 2023-12-14T18:26:14.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.757+0000 7fa0ce7fc700 1 -- 172.21.15.93:0/3070827146 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa0d0004bb0 con 0x7fa0d8100ca0 2023-12-14T18:26:14.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.757+0000 7fa0ce7fc700 1 -- 172.21.15.93:0/3070827146 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0d0005280 con 0x7fa0d8100ca0 2023-12-14T18:26:14.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.757+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0d810a8f0 con 0x7fa0d8100ca0 2023-12-14T18:26:14.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.757+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0d80679f0 con 0x7fa0d8100ca0 2023-12-14T18:26:14.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.759+0000 7fa0ce7fc700 1 -- 172.21.15.93:0/3070827146 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fa0d0004d20 con 0x7fa0d8100ca0 2023-12-14T18:26:14.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.759+0000 7fa0ce7fc700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0c40a1150 0x7fa0c40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:14.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.759+0000 7fa0dd4e6700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0c40a1150 0x7fa0c40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:14.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.759+0000 7fa0ce7fc700 1 -- 172.21.15.93:0/3070827146 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fa0d00cf3e0 con 0x7fa0d8100ca0 2023-12-14T18:26:14.763 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.760+0000 7fa0dd4e6700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0c40a1150 0x7fa0c40a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8005fd0 tx=0x7fa0c8005e50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:14.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.762+0000 7fa0ce7fc700 1 -- 172.21.15.93:0/3070827146 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa0d0098820 con 0x7fa0d8100ca0 2023-12-14T18:26:14.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.950+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7fa0d8068ab0 con 0x7fa0d8100ca0 2023-12-14T18:26:14.952 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.950+0000 7fa0ce7fc700 1 -- 172.21.15.93:0/3070827146 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v86) v1 ==== 103+0+11 (secure 0 0 0) 0x7fa0d0020020 con 0x7fa0d8100ca0 2023-12-14T18:26:14.952 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:26:14.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.952+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0c40a1150 msgr2=0x7fa0c40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.952+0000 7fa0df74a700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0c40a1150 0x7fa0c40a3600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa0c8005fd0 tx=0x7fa0c8005e50 comp rx=0 tx=0).stop 2023-12-14T18:26:14.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.952+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100ca0 msgr2=0x7fa0d819c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:14.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.952+0000 7fa0df74a700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100ca0 0x7fa0d819c3d0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fa0d0007f10 tx=0x7fa0d000f5c0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.953+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 shutdown_connections 2023-12-14T18:26:14.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.953+0000 7fa0df74a700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fa0c40a1150 0x7fa0c40a3600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.953+0000 7fa0df74a700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa0d8100ca0 0x7fa0d819c3d0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.955 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.953+0000 7fa0df74a700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa0d80ff120 0x7fa0d8194e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.953+0000 7fa0df74a700 1 --2- 172.21.15.93:0/3070827146 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa0d8100310 0x7fa0d8195360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:14.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.953+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 >> 172.21.15.93:0/3070827146 conn(0x7fa0d80fa740 msgr2=0x7fa0d80fc960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:14.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.953+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 shutdown_connections 2023-12-14T18:26:14.956 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:14.953+0000 7fa0df74a700 1 -- 172.21.15.93:0/3070827146 wait complete. 2023-12-14T18:26:14.967 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:26:15.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.086+0000 7f2af7f23700 1 -- 172.21.15.93:0/3314540176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af00fe4e0 msgr2=0x7f2af00fe8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.086+0000 7f2af7f23700 1 --2- 172.21.15.93:0/3314540176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af00fe4e0 0x7f2af00fe8f0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f2aec009990 tx=0x7f2aec009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.086+0000 7f2af7f23700 1 -- 172.21.15.93:0/3314540176 shutdown_connections 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 --2- 172.21.15.93:0/3314540176 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af00fe4e0 0x7f2af00fe8f0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 --2- 172.21.15.93:0/3314540176 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2af0101e40 0x7f2af0108700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 --2- 172.21.15.93:0/3314540176 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af01014b0 0x7f2af0101900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 -- 172.21.15.93:0/3314540176 >> 172.21.15.93:0/3314540176 conn(0x7f2af00f9e80 msgr2=0x7f2af00fc2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 -- 172.21.15.93:0/3314540176 shutdown_connections 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 -- 172.21.15.93:0/3314540176 wait complete. 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 Processor -- start 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 -- start start 2023-12-14T18:26:15.092 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2af00fe4e0 0x7f2af0198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af01014b0 0x7f2af0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af0101e40 0x7f2af019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af019fd90 con 0x7f2af0101e40 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af019fed0 con 0x7f2af01014b0 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af7f23700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2af01a0060 con 0x7f2af00fe4e0 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.087+0000 7f2af64c0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af0101e40 0x7f2af019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af64c0700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af0101e40 0x7f2af019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46600/0 (socket says 172.21.15.93:46600) 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af64c0700 1 -- 172.21.15.93:0/747423473 learned_addr learned my addr 172.21.15.93:0/747423473 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af5cbf700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2af00fe4e0 0x7f2af0198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.093 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af54be700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af01014b0 0x7f2af0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af64c0700 1 -- 172.21.15.93:0/747423473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2af00fe4e0 msgr2=0x7f2af0198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af64c0700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2af00fe4e0 0x7f2af0198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af64c0700 1 -- 172.21.15.93:0/747423473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af01014b0 msgr2=0x7f2af0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af64c0700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af01014b0 0x7f2af0198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af64c0700 1 -- 172.21.15.93:0/747423473 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2aec0095f0 con 0x7f2af0101e40 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af64c0700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af0101e40 0x7f2af019f7c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800ada0 tx=0x7f2ae800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2ae6ffd700 1 -- 172.21.15.93:0/747423473 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae800f070 con 0x7f2af0101e40 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2ae6ffd700 1 -- 172.21.15.93:0/747423473 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2ae8007b30 con 0x7f2af0101e40 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2af010bb90 con 0x7f2af0101e40 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2ae6ffd700 1 -- 172.21.15.93:0/747423473 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ae80053e0 con 0x7f2af0101e40 2023-12-14T18:26:15.094 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.088+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2af010ba00 con 0x7f2af0101e40 2023-12-14T18:26:15.095 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.089+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2af01928f0 con 0x7f2af0101e40 2023-12-14T18:26:15.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.090+0000 7f2ae6ffd700 1 -- 172.21.15.93:0/747423473 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2ae800c980 con 0x7f2af0101e40 2023-12-14T18:26:15.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.091+0000 7f2ae6ffd700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2adc0a1150 0x7f2adc0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.091+0000 7f2ae6ffd700 1 -- 172.21.15.93:0/747423473 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2ae80cc850 con 0x7f2af0101e40 2023-12-14T18:26:15.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.091+0000 7f2af5cbf700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2adc0a1150 0x7f2adc0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.091+0000 7f2af5cbf700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2adc0a1150 0x7f2adc0a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00b5c0 tx=0x7f2aec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.096 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.094+0000 7f2ae6ffd700 1 -- 172.21.15.93:0/747423473 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2ae8095d20 con 0x7f2af0101e40 2023-12-14T18:26:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:15 smithi093 ceph-mon[42070]: pgmap v275: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-12-14T18:26:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:15 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1156484907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:15 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/169325209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:15 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3070827146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:15.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.324+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2af0061930 con 0x7f2af0101e40 2023-12-14T18:26:15.326 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.324+0000 7f2ae6ffd700 1 -- 172.21.15.93:0/747423473 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+10799 (secure 0 0 0) 0x7f2ae8099340 con 0x7f2af0101e40 2023-12-14T18:26:15.326 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:26:15.326 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":24,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":24,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:25:01.760987+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.326+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2adc0a1150 msgr2=0x7f2adc0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.326+0000 7f2af7f23700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2adc0a1150 0x7f2adc0a3600 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f2aec00b5c0 tx=0x7f2aec005fd0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.326+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af0101e40 msgr2=0x7f2af019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.326+0000 7f2af7f23700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af0101e40 0x7f2af019f7c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f2ae800ada0 tx=0x7f2ae800c330 comp rx=0 tx=0).stop 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.326+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 shutdown_connections 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.327+0000 7f2af7f23700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2adc0a1150 0x7f2adc0a3600 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.327+0000 7f2af7f23700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2af0101e40 0x7f2af019f7c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.327+0000 7f2af7f23700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2af00fe4e0 0x7f2af0198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.327+0000 7f2af7f23700 1 --2- 172.21.15.93:0/747423473 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2af01014b0 0x7f2af0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.327+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 >> 172.21.15.93:0/747423473 conn(0x7f2af00f9e80 msgr2=0x7f2af0106e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.327+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 shutdown_connections 2023-12-14T18:26:15.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.327+0000 7f2af7f23700 1 -- 172.21.15.93:0/747423473 wait complete. 2023-12-14T18:26:15.330 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 24 2023-12-14T18:26:15.341 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:26:15.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.461+0000 7f648d69b700 1 -- 172.21.15.93:0/3581398549 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 msgr2=0x7f6488100790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.461+0000 7f648d69b700 1 --2- 172.21.15.93:0/3581398549 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 0x7f6488100790 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f6470009990 tx=0x7f6470009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.461+0000 7f648d69b700 1 -- 172.21.15.93:0/3581398549 shutdown_connections 2023-12-14T18:26:15.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.461+0000 7f648d69b700 1 --2- 172.21.15.93:0/3581398549 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6488100cd0 0x7f6488107590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.461+0000 7f648d69b700 1 --2- 172.21.15.93:0/3581398549 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f64880ff150 0x7f64880ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.461+0000 7f648d69b700 1 --2- 172.21.15.93:0/3581398549 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 0x7f6488100790 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.461+0000 7f648d69b700 1 -- 172.21.15.93:0/3581398549 >> 172.21.15.93:0/3581398549 conn(0x7f64880fa750 msgr2=0x7f64880fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.461+0000 7f648d69b700 1 -- 172.21.15.93:0/3581398549 shutdown_connections 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.461+0000 7f648d69b700 1 -- 172.21.15.93:0/3581398549 wait complete. 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f648d69b700 1 Processor -- start 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f648d69b700 1 -- start start 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f648d69b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64880ff150 0x7f6488069b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f648d69b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 0x7f648819d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f648d69b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488100cd0 0x7f648819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f648d69b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648819fd50 con 0x7f6488100340 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f648d69b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f648819fec0 con 0x7f64880ff150 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f648d69b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64881a0080 con 0x7f6488100cd0 2023-12-14T18:26:15.466 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f64867fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 0x7f648819d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f64867fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 0x7f648819d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46612/0 (socket says 172.21.15.93:46612) 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f64867fc700 1 -- 172.21.15.93:0/4278512348 learned_addr learned my addr 172.21.15.93:0/4278512348 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f64877fe700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488100cd0 0x7f648819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f6486ffd700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64880ff150 0x7f6488069b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f64867fc700 1 -- 172.21.15.93:0/4278512348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488100cd0 msgr2=0x7f648819f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f64867fc700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488100cd0 0x7f648819f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f64867fc700 1 -- 172.21.15.93:0/4278512348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64880ff150 msgr2=0x7f6488069b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f64867fc700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64880ff150 0x7f6488069b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f64867fc700 1 -- 172.21.15.93:0/4278512348 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64700095f0 con 0x7f6488100340 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.462+0000 7f6486ffd700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64880ff150 0x7f6488069b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.463+0000 7f64867fc700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 0x7f648819d150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f647000c010 tx=0x7f6470004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.463+0000 7f647ffff700 1 -- 172.21.15.93:0/4278512348 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f647001d070 con 0x7f6488100340 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.463+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f648810ab10 con 0x7f6488100340 2023-12-14T18:26:15.467 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.463+0000 7f647ffff700 1 -- 172.21.15.93:0/4278512348 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f64700054f0 con 0x7f6488100340 2023-12-14T18:26:15.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.463+0000 7f647ffff700 1 -- 172.21.15.93:0/4278512348 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6470004ca0 con 0x7f6488100340 2023-12-14T18:26:15.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.463+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f648810a980 con 0x7f6488100340 2023-12-14T18:26:15.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.464+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64881060d0 con 0x7f6488100340 2023-12-14T18:26:15.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.465+0000 7f647ffff700 1 -- 172.21.15.93:0/4278512348 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f647000fbc0 con 0x7f6488100340 2023-12-14T18:26:15.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.466+0000 7f647ffff700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f64740a1150 0x7f64740a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.466+0000 7f647ffff700 1 -- 172.21.15.93:0/4278512348 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f64700ce400 con 0x7f6488100340 2023-12-14T18:26:15.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.466+0000 7f6486ffd700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f64740a1150 0x7f64740a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.466+0000 7f6486ffd700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f64740a1150 0x7f64740a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f64881001a0 tx=0x7f647800b670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.469+0000 7f647ffff700 1 -- 172.21.15.93:0/4278512348 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6470097870 con 0x7f6488100340 2023-12-14T18:26:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:15 smithi160 ceph-mon[43840]: pgmap v275: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-12-14T18:26:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:15 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1156484907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:15 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/169325209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:15 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3070827146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:15.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:15 smithi097 ceph-mon[43486]: pgmap v275: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-12-14T18:26:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:15 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1156484907' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:26:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:15 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/169325209' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:26:15.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:15 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3070827146' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:26:15.660 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.658+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6488061700 con 0x7f6488100340 2023-12-14T18:26:15.661 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.660+0000 7f647ffff700 1 -- 172.21.15.93:0/4278512348 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) v1 ==== 74+0+24357 (secure 0 0 0) 0x7f647009ae90 con 0x7f6488100340 2023-12-14T18:26:15.662 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:26:15.662 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":86,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:25:37.836200+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"80","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:26:15.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f64740a1150 msgr2=0x7f64740a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f64740a1150 0x7f64740a3600 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f64881001a0 tx=0x7f647800b670 comp rx=0 tx=0).stop 2023-12-14T18:26:15.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 msgr2=0x7f648819d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 0x7f648819d150 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f647000c010 tx=0x7f6470004730 comp rx=0 tx=0).stop 2023-12-14T18:26:15.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 shutdown_connections 2023-12-14T18:26:15.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f64740a1150 0x7f64740a3600 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.664 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6488100cd0 0x7f648819f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f64880ff150 0x7f6488069b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 --2- 172.21.15.93:0/4278512348 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6488100340 0x7f648819d150 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 >> 172.21.15.93:0/4278512348 conn(0x7f64880fa750 msgr2=0x7f64880fca30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 shutdown_connections 2023-12-14T18:26:15.665 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.662+0000 7f648d69b700 1 -- 172.21.15.93:0/4278512348 wait complete. 2023-12-14T18:26:15.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2023-12-14T18:24:36.557875+0000', 'modified': '2023-12-14T18:25:01.760987+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 3, 'in': [0, 1, 2], 'up': {'mds_0': 24461, 'mds_1': 24442, 'mds_2': 24422}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6835/2285100381', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6834', 'nonce': 2285100381}, {'type': 'v1', 'addr': '172.21.15.160:6835', 'nonce': 2285100381}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24422': {'gid': 24422, 'name': 'e', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.97:6835/1146986690', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6834', 'nonce': 1146986690}, {'type': 'v1', 'addr': '172.21.15.97:6835', 'nonce': 1146986690}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24424': {'gid': 24424, 'name': 'c', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.160:6837/233011976', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6836', 'nonce': 233011976}, {'type': 'v1', 'addr': '172.21.15.160:6837', 'nonce': 233011976}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24440': {'gid': 24440, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.97:6837/3050538896', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6836', 'nonce': 3050538896}, {'type': 'v1', 'addr': '172.21.15.97:6837', 'nonce': 3050538896}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24442': {'gid': 24442, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 10, 'addr': '172.21.15.160:6839/3370815452', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.160:6838', 'nonce': 3370815452}, {'type': 'v1', 'addr': '172.21.15.160:6839', 'nonce': 3370815452}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}, 'gid_24461': {'gid': 24461, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.97:6839/3753662808', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.97:6838', 'nonce': 3753662808}, {'type': 'v1', 'addr': '172.21.15.97:6839', 'nonce': 3753662808}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-14T18:26:15.677 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 3/3 2023-12-14T18:26:15.677 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.e status 2023-12-14T18:26:15.803 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.799+0000 7f6ef3788700 1 -- 172.21.15.93:0/1527027517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec0ff160 msgr2=0x7f6eec0ff570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.799+0000 7f6ef3788700 1 --2- 172.21.15.93:0/1527027517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec0ff160 0x7f6eec0ff570 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f6edc00b0a0 tx=0x7f6edc00b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.799+0000 7f6ef3788700 1 -- 172.21.15.93:0/1527027517 shutdown_connections 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.799+0000 7f6ef3788700 1 --2- 172.21.15.93:0/1527027517 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec0ff160 0x7f6eec0ff570 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.799+0000 7f6ef3788700 1 --2- 172.21.15.93:0/1527027517 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec100ce0 0x7f6eec1075a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.799+0000 7f6ef3788700 1 --2- 172.21.15.93:0/1527027517 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6eec100350 0x7f6eec1007a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.799+0000 7f6ef3788700 1 -- 172.21.15.93:0/1527027517 >> 172.21.15.93:0/1527027517 conn(0x7f6eec0fa780 msgr2=0x7f6eec0fcbb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.799+0000 7f6ef3788700 1 -- 172.21.15.93:0/1527027517 shutdown_connections 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.799+0000 7f6ef3788700 1 -- 172.21.15.93:0/1527027517 wait complete. 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef3788700 1 Processor -- start 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef3788700 1 -- start start 2023-12-14T18:26:15.804 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef3788700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0ff160 0x7f6eec196100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef3788700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec100350 0x7f6eec196640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef3788700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6eec100ce0 0x7f6eec19d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef3788700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec19dc80 con 0x7f6eec100350 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef3788700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec19ddc0 con 0x7f6eec0ff160 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef3788700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6eec19df50 con 0x7f6eec100ce0 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef0d23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec100350 0x7f6eec196640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef0d23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec100350 0x7f6eec196640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46626/0 (socket says 172.21.15.93:46626) 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef0d23700 1 -- 172.21.15.93:0/869713844 learned_addr learned my addr 172.21.15.93:0/869713844 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef1524700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0ff160 0x7f6eec196100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef1d25700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6eec100ce0 0x7f6eec19d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef0d23700 1 -- 172.21.15.93:0/869713844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6eec100ce0 msgr2=0x7f6eec19d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.805 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6eec100ce0 0x7f6eec19d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef0d23700 1 -- 172.21.15.93:0/869713844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0ff160 msgr2=0x7f6eec196100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0ff160 0x7f6eec196100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef0d23700 1 -- 172.21.15.93:0/869713844 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6edc009d00 con 0x7f6eec100350 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef1524700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0ff160 0x7f6eec196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.800+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec100350 0x7f6eec196640 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000e9a0 tx=0x7f6ee000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.801+0000 7f6eea7fc700 1 -- 172.21.15.93:0/869713844 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0009790 con 0x7f6eec100350 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.801+0000 7f6eea7fc700 1 -- 172.21.15.93:0/869713844 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee000cbd0 con 0x7f6eec100350 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.801+0000 7f6ef3788700 1 -- 172.21.15.93:0/869713844 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6eec10bc40 con 0x7f6eec100350 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.801+0000 7f6eea7fc700 1 -- 172.21.15.93:0/869713844 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0018680 con 0x7f6eec100350 2023-12-14T18:26:15.806 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.801+0000 7f6ef3788700 1 -- 172.21.15.93:0/869713844 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec10bab0 con 0x7f6eec100350 2023-12-14T18:26:15.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.803+0000 7f6eea7fc700 1 -- 172.21.15.93:0/869713844 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f6ee000cd40 con 0x7f6eec100350 2023-12-14T18:26:15.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.804+0000 7f6eea7fc700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6ed40a1150 0x7f6ed40a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.804+0000 7f6eea7fc700 1 -- 172.21.15.93:0/869713844 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ee0014070 con 0x7f6eec100350 2023-12-14T18:26:15.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.804+0000 7f6ef1524700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6ed40a1150 0x7f6ed40a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.804+0000 7f6ef1524700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6ed40a1150 0x7f6ed40a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0079f0 tx=0x7f6edc009c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6eff916b80 1 Processor -- start 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6eff916b80 1 -- start start 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 0x55b125b6a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6ae00 0x55b125b6b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b6c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6eff916b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125a53590 con 0x55b125b6ae00 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6eff916b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125ac2820 con 0x55b125b6bcb0 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6eff916b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125abbcb0 con 0x55b125b6a4b0 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef0d23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6ae00 0x55b125b6b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef0d23700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6ae00 0x55b125b6b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46628/0 (socket says 172.21.15.93:46628) 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef0d23700 1 -- 172.21.15.93:0/1972619298 learned_addr learned my addr 172.21.15.93:0/1972619298 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef1d25700 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b6c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef1524700 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 0x55b125b6a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef0d23700 1 -- 172.21.15.93:0/1972619298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 msgr2=0x55b125b6a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 0x55b125b6a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef0d23700 1 -- 172.21.15.93:0/1972619298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 msgr2=0x55b125b6c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b6c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef0d23700 1 -- 172.21.15.93:0/1972619298 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b12593d360 con 0x55b125b6ae00 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.805+0000 7f6ef1524700 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 0x55b125b6a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6ae00 0x55b125b6b720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00a1fc0 tx=0x7f6ee009a0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=502ef67135b06882 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6ed37fe700 1 -- 172.21.15.93:0/1972619298 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00a24f0 con 0x55b125b6ae00 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6ed37fe700 1 -- 172.21.15.93:0/1972619298 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee0098440 con 0x55b125b6ae00 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6ed37fe700 1 -- 172.21.15.93:0/1972619298 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee0099260 con 0x55b125b6ae00 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6eff916b80 1 -- 172.21.15.93:0/1972619298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6ae00 msgr2=0x55b125b6b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6eff916b80 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6ae00 0x55b125b6b720 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00a1fc0 tx=0x7f6ee009a0a0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6eff916b80 1 -- 172.21.15.93:0/1972619298 shutdown_connections 2023-12-14T18:26:15.816 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6eff916b80 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b6c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6eff916b80 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 0x55b125b6a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6eff916b80 1 --2- 172.21.15.93:0/1972619298 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6ae00 0x55b125b6b720 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6eff916b80 1 -- 172.21.15.93:0/1972619298 >> 172.21.15.93:0/1972619298 conn(0x55b125b5fc50 msgr2=0x55b125b653e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6eff916b80 1 -- 172.21.15.93:0/1972619298 shutdown_connections 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.806+0000 7f6eff916b80 1 -- 172.21.15.93:0/1972619298 wait complete. 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6eff916b80 1 -- start start 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6eff916b80 1 Processor -- start 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b74150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b74690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.817 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b7b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6eff916b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125a57f40 con 0x55b125b6a4b0 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6eff916b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125a57be0 con 0x55b125b6bcb0 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6eff916b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b12593f650 con 0x55b125b6ae00 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6eff916b80 10 client.0 _ll_register_callbacks cb 0x55b125a7b8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6ef1524700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b74150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6ef0d23700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b74690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6ef1524700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b74150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46630/0 (socket says 172.21.15.93:46630) 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6ef1d25700 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b7b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.807+0000 7f6ef1524700 1 -- 172.21.15.93:0/4175122708 learned_addr learned my addr 172.21.15.93:0/4175122708 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ef0d23700 1 -- 172.21.15.93:0/4175122708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 msgr2=0x55b125b7b6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b7b6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ef0d23700 1 -- 172.21.15.93:0/4175122708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 msgr2=0x55b125b74150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.818 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b74150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ef0d23700 1 -- 172.21.15.93:0/4175122708 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec10bab0 con 0x55b125b6ae00 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ef1d25700 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b7b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ef1524700 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b74150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b74690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00a1fc0 tx=0x7f6ee009e250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cecf08fbb974afd2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ed0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/4175122708 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee00960a0 con 0x55b125b6ae00 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6eff916b80 1 -- 172.21.15.93:0/4175122708 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b125b5d260 con 0x55b125b6ae00 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/4175122708 <== mon.1 v2:172.21.15.160:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ee0099000 con 0x55b125b6ae00 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/4175122708 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee0096200 con 0x55b125b6ae00 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6eff916b80 10 client.25039 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6eff916b80 1 -- 172.21.15.93:0/4175122708 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b125a82120 con 0x55b125b6ae00 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.808+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/4175122708 <== mon.1 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55b125b5d260 con 0x55b125b6ae00 2023-12-14T18:26:15.819 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/4175122708 <== mon.1 v2:172.21.15.160:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ee009afa0 con 0x55b125b6ae00 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6eff916b80 10 client.25039 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6eff916b80 10 client.25039 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6eff916b80 20 client.25039 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x55b125b7ef20 0x55b125b7e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6eff916b80 4 client.25039 mds_command: new command op to 24422 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6eff916b80 1 -- 172.21.15.93:0/4175122708 --> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b125ab4c90 con 0x55b125b7ef20 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6ef1524700 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x55b125b7ef20 0x55b125b7e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6ed0ff9700 10 client.25039 ms_handle_connect on v2:172.21.15.97:6834/1146986690 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.809+0000 7f6ef1524700 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x55b125b7ef20 0x55b125b7e570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7b250c858e1fb5a6 server_cookie=b2ec22360dc64a54 in_seq=0 out_seq=0 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/4175122708 <== mds.2 v2:172.21.15.97:6834/1146986690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55b125ab4c90 con 0x55b125b7ef20 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6ed0ff9700 10 client.25039 handle_command_reply: tid=0 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 client.25039 shutdown 2023-12-14T18:26:15.820 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 -- 172.21.15.93:0/4175122708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 msgr2=0x55b125b74690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b74690 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6ee00a1fc0 tx=0x7f6ee009e250 comp rx=0 tx=0).stop 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 -- 172.21.15.93:0/4175122708 shutdown_connections 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x55b125b7ef20 0x55b125b7e570 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b74150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6ed0ff9700 0 client.25039 ms_handle_reset on v2:172.21.15.97:6834/1146986690 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b7b6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4175122708 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b74690 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 -- 172.21.15.93:0/4175122708 >> 172.21.15.93:0/4175122708 conn(0x55b125b5fc50 msgr2=0x55b125b63ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 -- 172.21.15.93:0/4175122708 shutdown_connections 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.810+0000 7f6eff916b80 1 -- 172.21.15.93:0/4175122708 wait complete. 2023-12-14T18:26:15.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.811+0000 7f6eff916b80 20 client.25039 trim_cache size 0 max 16384 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6eff916b80 1 Processor -- start 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6eff916b80 1 -- start start 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b7e570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b7b5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b72e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6eff916b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125b75340 con 0x55b125b6a4b0 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6eff916b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125ab7da0 con 0x55b125b6bcb0 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6eff916b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125a65580 con 0x55b125b6ae00 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1524700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b7e570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1524700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b7e570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46636/0 (socket says 172.21.15.93:46636) 2023-12-14T18:26:15.836 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1524700 1 -- 172.21.15.93:0/4064015049 learned_addr learned my addr 172.21.15.93:0/4064015049 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b7b5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1d25700 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b72e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1524700 1 -- 172.21.15.93:0/4064015049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 msgr2=0x55b125b7b5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1524700 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b7b5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1524700 1 -- 172.21.15.93:0/4064015049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 msgr2=0x55b125b72e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1524700 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b72e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1524700 1 -- 172.21.15.93:0/4064015049 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b125a81e70 con 0x55b125b6a4b0 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b7b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.828+0000 7f6ef1524700 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b7e570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009cd0 tx=0x7f6edc01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c44d35b240fb95 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6ed3fff700 1 -- 172.21.15.93:0/4064015049 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc00ba50 con 0x55b125b6a4b0 2023-12-14T18:26:15.837 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6ed3fff700 1 -- 172.21.15.93:0/4064015049 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6edc02d510 con 0x55b125b6a4b0 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6ed3fff700 1 -- 172.21.15.93:0/4064015049 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6edc02a7e0 con 0x55b125b6a4b0 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 -- 172.21.15.93:0/4064015049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 msgr2=0x55b125b7e570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b7e570 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6edc009cd0 tx=0x7f6edc01b000 comp rx=0 tx=0).stop 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 -- 172.21.15.93:0/4064015049 shutdown_connections 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6a4b0 0x55b125b7e570 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6bcb0 0x55b125b72e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 --2- 172.21.15.93:0/4064015049 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6ae00 0x55b125b7b5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 -- 172.21.15.93:0/4064015049 >> 172.21.15.93:0/4064015049 conn(0x55b125b5fc50 msgr2=0x55b125b65230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 -- 172.21.15.93:0/4064015049 shutdown_connections 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 -- 172.21.15.93:0/4064015049 wait complete. 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.829+0000 7f6eff916b80 1 -- start start 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6eff916b80 1 Processor -- start 2023-12-14T18:26:15.838 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 0x55b125b76880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6ae00 0x55b125b7b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6eff916b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6bcb0 0x55b125b76eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6eff916b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125bd2d80 con 0x55b125b6bcb0 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6eff916b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125bd2ec0 con 0x55b125b6ae00 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6eff916b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b125bd3050 con 0x55b125b6a4b0 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6eff916b80 10 client.0 _ll_register_callbacks cb 0x7f6ee0099a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1d25700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6bcb0 0x55b125b76eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1d25700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6bcb0 0x55b125b76eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46642/0 (socket says 172.21.15.93:46642) 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1d25700 1 -- 172.21.15.93:0/988414090 learned_addr learned my addr 172.21.15.93:0/988414090 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6ae00 0x55b125b7b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1524700 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 0x55b125b76880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1d25700 1 -- 172.21.15.93:0/988414090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 msgr2=0x55b125b76880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.839 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1d25700 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 0x55b125b76880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1d25700 1 -- 172.21.15.93:0/988414090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6ae00 msgr2=0x55b125b7b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1d25700 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6ae00 0x55b125b7b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1d25700 1 -- 172.21.15.93:0/988414090 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6eec068da0 con 0x55b125b6bcb0 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef0d23700 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6ae00 0x55b125b7b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ef1d25700 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6bcb0 0x55b125b76eb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4014970 tx=0x7f6ee4014c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c847b6c09ad7fc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.830+0000 7f6ed0ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/988414090 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee4019070 con 0x55b125b6bcb0 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/988414090 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6ee4004050 con 0x55b125b6bcb0 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6eff916b80 1 -- 172.21.15.93:0/988414090 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6ee0098440 con 0x55b125b6bcb0 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/988414090 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6ee4004210 con 0x55b125b6bcb0 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6eff916b80 10 client.15405 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6eff916b80 1 -- 172.21.15.93:0/988414090 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55b125a68950 con 0x55b125b6bcb0 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/988414090 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6ee0098440 con 0x55b125b6bcb0 2023-12-14T18:26:15.840 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/988414090 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f6ee4003840 con 0x55b125b6bcb0 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6eff916b80 10 client.15405 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6eff916b80 10 client.15405 resolve_mds: resolved name 'e' to daemon mds.e 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6eff916b80 20 client.15405 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6eff916b80 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x55b125bd7e80 0x55b125bd48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6eff916b80 4 client.15405 mds_command: new command op to 24422 tid=0[{"prefix": "status"}] 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6eff916b80 1 -- 172.21.15.93:0/988414090 --> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b125ab49b0 con 0x55b125bd7e80 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.831+0000 7f6ef1524700 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x55b125bd7e80 0x55b125bd48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6ef1524700 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x55b125bd7e80 0x55b125bd48b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3f1a84ca4f9ec610 server_cookie=fcfd40b6b957fb2 in_seq=0 out_seq=0 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6ed0ff9700 10 client.15405 ms_handle_connect on v2:172.21.15.97:6834/1146986690 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6ed0ff9700 1 -- 172.21.15.93:0/988414090 <== mds.2 v2:172.21.15.97:6834/1146986690 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55b125ab49b0 con 0x55b125bd7e80 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6ed0ff9700 10 client.15405 handle_command_reply: tid=0 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 client.15405 shutdown 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 -- 172.21.15.93:0/988414090 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6bcb0 msgr2=0x55b125b76eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.841 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6bcb0 0x55b125b76eb0 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f6ee4014970 tx=0x7f6ee4014c80 comp rx=0 tx=0).stop 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 -- 172.21.15.93:0/988414090 shutdown_connections 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.97:6834/1146986690,v1:172.21.15.97:6835/1146986690] conn(0x55b125bd7e80 0x55b125bd48b0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55b125b6bcb0 0x55b125b76eb0 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6ed0ff9700 0 client.15405 ms_handle_reset on v2:172.21.15.97:6834/1146986690 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55b125b6a4b0 0x55b125b76880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 --2- 172.21.15.93:0/988414090 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55b125b6ae00 0x55b125b7b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 -- 172.21.15.93:0/988414090 >> 172.21.15.93:0/988414090 conn(0x55b125b5fc50 msgr2=0x55b125b72e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 -- 172.21.15.93:0/988414090 shutdown_connections 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 1 -- 172.21.15.93:0/988414090 wait complete. 2023-12-14T18:26:15.842 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.832+0000 7f6eff916b80 20 client.15405 trim_cache size 0 max 16384 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 2, 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24422, 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 75.075818327999997, 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 24, 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 86, 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 84, 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 117.261358406 2023-12-14T18:26:15.843 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:26:15.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 -- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6ed40a1150 msgr2=0x7f6ed40a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6ed40a1150 0x7f6ed40a3600 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6edc0079f0 tx=0x7f6edc009c60 comp rx=0 tx=0).stop 2023-12-14T18:26:15.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 -- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec100350 msgr2=0x7f6eec196640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec100350 0x7f6eec196640 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000e9a0 tx=0x7f6ee000ed60 comp rx=0 tx=0).stop 2023-12-14T18:26:15.844 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 -- 172.21.15.93:0/869713844 shutdown_connections 2023-12-14T18:26:15.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6ed40a1150 0x7f6ed40a3600 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6eec100ce0 0x7f6eec19d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6eec0ff160 0x7f6eec196100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 --2- 172.21.15.93:0/869713844 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6eec100350 0x7f6eec196640 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 -- 172.21.15.93:0/869713844 >> 172.21.15.93:0/869713844 conn(0x7f6eec0fa780 msgr2=0x7f6eec0fc950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 -- 172.21.15.93:0/869713844 shutdown_connections 2023-12-14T18:26:15.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.833+0000 7f6ef3788700 1 -- 172.21.15.93:0/869713844 wait complete. 2023-12-14T18:26:15.847 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2023-12-14T18:26:15.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.968+0000 7f2b2f902700 1 -- 172.21.15.93:0/2728911964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b28100330 msgr2=0x7f2b28100780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.968+0000 7f2b2f902700 1 --2- 172.21.15.93:0/2728911964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b28100330 0x7f2b28100780 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f2b24009990 tx=0x7f2b24009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.968+0000 7f2b2f902700 1 -- 172.21.15.93:0/2728911964 shutdown_connections 2023-12-14T18:26:15.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.968+0000 7f2b2f902700 1 --2- 172.21.15.93:0/2728911964 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28100cc0 0x7f2b28107580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.968+0000 7f2b2f902700 1 --2- 172.21.15.93:0/2728911964 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b280ff140 0x7f2b280ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.968+0000 7f2b2f902700 1 --2- 172.21.15.93:0/2728911964 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b28100330 0x7f2b28100780 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.968+0000 7f2b2f902700 1 -- 172.21.15.93:0/2728911964 >> 172.21.15.93:0/2728911964 conn(0x7f2b280fa740 msgr2=0x7f2b280fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.968+0000 7f2b2f902700 1 -- 172.21.15.93:0/2728911964 shutdown_connections 2023-12-14T18:26:15.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.968+0000 7f2b2f902700 1 -- 172.21.15.93:0/2728911964 wait complete. 2023-12-14T18:26:15.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2f902700 1 Processor -- start 2023-12-14T18:26:15.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2f902700 1 -- start start 2023-12-14T18:26:15.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2f902700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b280ff140 0x7f2b28196010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2f902700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b28100330 0x7f2b28196550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2f902700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28100cc0 0x7f2b2819d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2f902700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2819db90 con 0x7f2b280ff140 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2f902700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2819dcd0 con 0x7f2b28100cc0 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2f902700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b2819de60 con 0x7f2b28100330 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2d69e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b280ff140 0x7f2b28196010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2d69e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b280ff140 0x7f2b28196010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46660/0 (socket says 172.21.15.93:46660) 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2d69e700 1 -- 172.21.15.93:0/572237792 learned_addr learned my addr 172.21.15.93:0/572237792 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b28100330 0x7f2b28196550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.969+0000 7f2b2de9f700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28100cc0 0x7f2b2819d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b2d69e700 1 -- 172.21.15.93:0/572237792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b28100330 msgr2=0x7f2b28196550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b28100330 0x7f2b28196550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b2d69e700 1 -- 172.21.15.93:0/572237792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28100cc0 msgr2=0x7f2b2819d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28100cc0 0x7f2b2819d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b2d69e700 1 -- 172.21.15.93:0/572237792 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b240095f0 con 0x7f2b280ff140 2023-12-14T18:26:15.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b2de9f700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28100cc0 0x7f2b2819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:15.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b280ff140 0x7f2b28196010 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800d980 tx=0x7f2b1800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b1e7fc700 1 -- 172.21.15.93:0/572237792 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b1800de60 con 0x7f2b280ff140 2023-12-14T18:26:15.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b2f902700 1 -- 172.21.15.93:0/572237792 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b2810aaa0 con 0x7f2b280ff140 2023-12-14T18:26:15.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b1e7fc700 1 -- 172.21.15.93:0/572237792 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b180044e0 con 0x7f2b280ff140 2023-12-14T18:26:15.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b1e7fc700 1 -- 172.21.15.93:0/572237792 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18003be0 con 0x7f2b280ff140 2023-12-14T18:26:15.975 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.970+0000 7f2b2f902700 1 -- 172.21.15.93:0/572237792 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b2810a910 con 0x7f2b280ff140 2023-12-14T18:26:15.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.972+0000 7f2b1e7fc700 1 -- 172.21.15.93:0/572237792 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f2b18009bb0 con 0x7f2b280ff140 2023-12-14T18:26:15.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.973+0000 7f2b1e7fc700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b140a1150 0x7f2b140a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.973+0000 7f2b1e7fc700 1 -- 172.21.15.93:0/572237792 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2b180cc9b0 con 0x7f2b280ff140 2023-12-14T18:26:15.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.973+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b140a1150 0x7f2b140a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b140a1150 0x7f2b140a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2b24006010 tx=0x7f2b2400b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b3ba90b80 1 Processor -- start 2023-12-14T18:26:15.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b3ba90b80 1 -- start start 2023-12-14T18:26:15.982 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635148488c0 0x563514848cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563514849210 0x563514849b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 0x56351484a530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56351472d590 con 0x563514849210 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56351479c820 con 0x5635148488c0 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563514795cb0 con 0x56351484a0c0 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2ce9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563514849210 0x563514849b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2ce9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563514849210 0x563514849b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46662/0 (socket says 172.21.15.93:46662) 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2ce9d700 1 -- 172.21.15.93:0/3130666859 learned_addr learned my addr 172.21.15.93:0/3130666859 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2de9f700 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 0x56351484a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635148488c0 0x563514848cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2ce9d700 1 -- 172.21.15.93:0/3130666859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 msgr2=0x56351484a530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 0x56351484a530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2ce9d700 1 -- 172.21.15.93:0/3130666859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635148488c0 msgr2=0x563514848cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635148488c0 0x563514848cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.983 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2ce9d700 1 -- 172.21.15.93:0/3130666859 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563514617360 con 0x563514849210 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635148488c0 0x563514848cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.974+0000 7f2b2de9f700 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 0x56351484a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563514849210 0x563514849b30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2b24005360 tx=0x7f2b2401f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2992e0aaad6cdeba server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b137fe700 1 -- 172.21.15.93:0/3130666859 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b2402b070 con 0x563514849210 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b137fe700 1 -- 172.21.15.93:0/3130666859 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b2401e040 con 0x563514849210 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b137fe700 1 -- 172.21.15.93:0/3130666859 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b24004c10 con 0x563514849210 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/3130666859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563514849210 msgr2=0x563514849b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563514849210 0x563514849b30 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f2b24005360 tx=0x7f2b2401f000 comp rx=0 tx=0).stop 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/3130666859 shutdown_connections 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 0x56351484a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635148488c0 0x563514848cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/3130666859 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563514849210 0x563514849b30 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.975+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/3130666859 >> 172.21.15.93:0/3130666859 conn(0x563514839c50 msgr2=0x56351483f3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/3130666859 shutdown_connections 2023-12-14T18:26:15.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/3130666859 wait complete. 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 -- start start 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 Processor -- start 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514852450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563514849210 0x563514852990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56351484a0c0 0x5635148599d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563514731f40 con 0x5635148488c0 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563514731be0 con 0x56351484a0c0 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563514619650 con 0x563514849210 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b3ba90b80 10 client.0 _ll_register_callbacks cb 0x5635147558c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b2d69e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514852450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b2d69e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514852450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46678/0 (socket says 172.21.15.93:46678) 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.976+0000 7f2b2d69e700 1 -- 172.21.15.93:0/2890358011 learned_addr learned my addr 172.21.15.93:0/2890358011 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563514849210 0x563514852990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b2de9f700 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56351484a0c0 0x5635148599d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b2d69e700 1 -- 172.21.15.93:0/2890358011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563514849210 msgr2=0x563514852990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563514849210 0x563514852990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b2d69e700 1 -- 172.21.15.93:0/2890358011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56351484a0c0 msgr2=0x5635148599d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56351484a0c0 0x5635148599d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b2d69e700 1 -- 172.21.15.93:0/2890358011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b28107eb0 con 0x5635148488c0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514852450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b18099a40 tx=0x7f2b18099b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fcc49e8bae10137 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2890358011 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b18098fa0 con 0x5635148488c0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2890358011 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2b18099100 con 0x5635148488c0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2890358011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563514837260 con 0x5635148488c0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2890358011 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b180a1a70 con 0x5635148488c0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b3ba90b80 10 client.15417 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2890358011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56351475c120 con 0x5635148488c0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2890358011 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563514837260 con 0x5635148488c0 2023-12-14T18:26:15.986 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2890358011 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b18098ba0 con 0x5635148488c0 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b3ba90b80 10 client.15417 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b3ba90b80 10 client.15417 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.977+0000 7f2b3ba90b80 20 client.15417 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.978+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x56351485d510 0x56351485c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.978+0000 7f2b3ba90b80 4 client.15417 mds_command: new command op to 24442 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.978+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2890358011 --> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56351478ec90 con 0x56351485d510 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.978+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x56351485d510 0x56351485c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.978+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x56351485d510 0x56351485c9f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b6dae2585d1b7fd6 server_cookie=440afca54a76c316 in_seq=0 out_seq=0 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.978+0000 7f2b10ff9700 10 client.15417 ms_handle_connect on v2:172.21.15.160:6838/3370815452 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2890358011 <== mds.1 v2:172.21.15.160:6838/3370815452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56351478ec90 con 0x56351485d510 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b10ff9700 10 client.15417 handle_command_reply: tid=0 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 client.15417 shutdown 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2890358011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 msgr2=0x563514852450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:15.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514852450 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f2b18099a40 tx=0x7f2b18099b60 comp rx=0 tx=0).stop 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2890358011 shutdown_connections 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514852450 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x56351485d510 0x56351485c9f0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56351484a0c0 0x5635148599d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b10ff9700 0 client.15417 ms_handle_reset on v2:172.21.15.160:6838/3370815452 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2890358011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563514849210 0x563514852990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2890358011 >> 172.21.15.93:0/2890358011 conn(0x563514839c50 msgr2=0x56351483dee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2890358011 shutdown_connections 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2890358011 wait complete. 2023-12-14T18:26:15.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.979+0000 7f2b3ba90b80 20 client.15417 trim_cache size 0 max 16384 2023-12-14T18:26:16.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b3ba90b80 1 Processor -- start 2023-12-14T18:26:16.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b3ba90b80 1 -- start start 2023-12-14T18:26:16.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5635148488c0 0x56351483e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 0x563514859990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56351484a0c0 0x563514850490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56351473f580 con 0x56351484a0c0 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563514754290 con 0x563514849210 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635148484c0 con 0x5635148488c0 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2de9f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56351484a0c0 0x563514850490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2de9f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56351484a0c0 0x563514850490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46688/0 (socket says 172.21.15.93:46688) 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2de9f700 1 -- 172.21.15.93:0/1775046770 learned_addr learned my addr 172.21.15.93:0/1775046770 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5635148488c0 0x56351483e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 0x563514859990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2de9f700 1 -- 172.21.15.93:0/1775046770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5635148488c0 msgr2=0x56351483e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2de9f700 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5635148488c0 0x56351483e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2de9f700 1 -- 172.21.15.93:0/1775046770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 msgr2=0x563514859990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2de9f700 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 0x563514859990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2de9f700 1 -- 172.21.15.93:0/1775046770 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56351475be70 con 0x56351484a0c0 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.997+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 0x563514859990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b2de9f700 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56351484a0c0 0x563514850490 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009b70 tx=0x7f2b20009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1dd462424de7ca2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b13fff700 1 -- 172.21.15.93:0/1775046770 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b20004bb0 con 0x56351484a0c0 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b13fff700 1 -- 172.21.15.93:0/1775046770 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b20007dd0 con 0x56351484a0c0 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b13fff700 1 -- 172.21.15.93:0/1775046770 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b200056a0 con 0x56351484a0c0 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/1775046770 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56351484a0c0 msgr2=0x563514850490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56351484a0c0 0x563514850490 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f2b20009b70 tx=0x7f2b20009040 comp rx=0 tx=0).stop 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/1775046770 shutdown_connections 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56351484a0c0 0x563514850490 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5635148488c0 0x56351483e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/1775046770 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 0x563514859990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/1775046770 >> 172.21.15.93:0/1775046770 conn(0x563514839c50 msgr2=0x5635148511e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/1775046770 shutdown_connections 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.998+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/1775046770 wait complete. 2023-12-14T18:26:16.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b3ba90b80 1 -- start start 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b3ba90b80 1 Processor -- start 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514854db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 0x5635148552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b3ba90b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 0x563514858310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563514858850 con 0x5635148488c0 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635148589c0 con 0x563514849210 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b3ba90b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563514858b80 con 0x56351484a0c0 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b3ba90b80 10 client.0 _ll_register_callbacks cb 0x7f2b18099410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b2d69e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514854db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b2d69e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514854db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46700/0 (socket says 172.21.15.93:46700) 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b2d69e700 1 -- 172.21.15.93:0/2254469 learned_addr learned my addr 172.21.15.93:0/2254469 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:15.999+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 0x5635148552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b2d69e700 1 -- 172.21.15.93:0/2254469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 msgr2=0x563514858310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b2de9f700 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 0x563514858310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 0x563514858310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b2d69e700 1 -- 172.21.15.93:0/2254469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 msgr2=0x5635148552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 0x5635148552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b2d69e700 1 -- 172.21.15.93:0/2254469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b20011040 con 0x5635148488c0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b2d69e700 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514854db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2b180a19d0 tx=0x7f2b1800d8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38e4c809047f2b42 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b10ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2254469 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b180a1c70 con 0x5635148488c0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2254469 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f2b18098fa0 con 0x5635148488c0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2254469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2b2401e040 con 0x5635148488c0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2254469 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2b180b9200 con 0x5635148488c0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b3ba90b80 10 client.15423 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2254469 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563514742950 con 0x5635148488c0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2254469 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2b2401e040 con 0x5635148488c0 2023-12-14T18:26:16.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2254469 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f2b18098b30 con 0x5635148488c0 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b3ba90b80 10 client.15423 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b3ba90b80 10 client.15423 resolve_mds: resolved name 'i' to daemon mds.i 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b3ba90b80 20 client.15423 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x5635148b6a80 0x5635148b33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b3ba90b80 4 client.15423 mds_command: new command op to 24442 tid=0[{"prefix": "status"}] 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.000+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2254469 --> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56351478e9b0 con 0x5635148b6a80 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.001+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x5635148b6a80 0x5635148b33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.001+0000 7f2b10ff9700 10 client.15423 ms_handle_connect on v2:172.21.15.160:6838/3370815452 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.001+0000 7f2b2ce9d700 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x5635148b6a80 0x5635148b33d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=93f84990d36cda3d server_cookie=466f1c7f37b81fb in_seq=0 out_seq=0 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.001+0000 7f2b10ff9700 1 -- 172.21.15.93:0/2254469 <== mds.1 v2:172.21.15.160:6838/3370815452 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x56351478e9b0 con 0x5635148b6a80 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.001+0000 7f2b10ff9700 10 client.15423 handle_command_reply: tid=0 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.001+0000 7f2b3ba90b80 1 client.15423 shutdown 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2254469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 msgr2=0x563514854db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514854db0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f2b180a19d0 tx=0x7f2b1800d8d0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2254469 shutdown_connections 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635148488c0 0x563514854db0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.160:6838/3370815452,v1:172.21.15.160:6839/3370815452] conn(0x5635148b6a80 0x5635148b33d0 crc :-1 s=READY pgs=22 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56351484a0c0 0x563514858310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b10ff9700 0 client.15423 ms_handle_reset on v2:172.21.15.160:6838/3370815452 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 --2- 172.21.15.93:0/2254469 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563514849210 0x5635148552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2254469 >> 172.21.15.93:0/2254469 conn(0x563514839c50 msgr2=0x56351485c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2254469 shutdown_connections 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 1 -- 172.21.15.93:0/2254469 wait complete. 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b3ba90b80 20 client.15423 trim_cache size 0 max 16384 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:26:16.010 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 1, 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24442, 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 75.973550767000006, 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 24, 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 86, 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 84, 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 110.143403382 2023-12-14T18:26:16.011 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:26:16.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 -- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b140a1150 msgr2=0x7f2b140a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b140a1150 0x7f2b140a3600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f2b24006010 tx=0x7f2b2400b540 comp rx=0 tx=0).stop 2023-12-14T18:26:16.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 -- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b280ff140 msgr2=0x7f2b28196010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b280ff140 0x7f2b28196010 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f2b1800d980 tx=0x7f2b1800dd40 comp rx=0 tx=0).stop 2023-12-14T18:26:16.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 -- 172.21.15.93:0/572237792 shutdown_connections 2023-12-14T18:26:16.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f2b140a1150 0x7f2b140a3600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2b280ff140 0x7f2b28196010 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2b28100cc0 0x7f2b2819d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.012 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 --2- 172.21.15.93:0/572237792 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2b28100330 0x7f2b28196550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 -- 172.21.15.93:0/572237792 >> 172.21.15.93:0/572237792 conn(0x7f2b280fa740 msgr2=0x7f2b28068e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:16.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 -- 172.21.15.93:0/572237792 shutdown_connections 2023-12-14T18:26:16.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.002+0000 7f2b2f902700 1 -- 172.21.15.93:0/572237792 wait complete. 2023-12-14T18:26:16.016 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-14T18:26:16.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.138+0000 7f8c3d775700 1 -- 172.21.15.93:0/329909999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 msgr2=0x7f8c381086a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.138+0000 7f8c3d775700 1 --2- 172.21.15.93:0/329909999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 0x7f8c381086a0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000b0a0 tx=0x7f8c3000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.138+0000 7f8c3d775700 1 -- 172.21.15.93:0/329909999 shutdown_connections 2023-12-14T18:26:16.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.138+0000 7f8c3d775700 1 --2- 172.21.15.93:0/329909999 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 0x7f8c381086a0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.138+0000 7f8c3d775700 1 --2- 172.21.15.93:0/329909999 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c38100260 0x7f8c38100670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.138+0000 7f8c3d775700 1 --2- 172.21.15.93:0/329909999 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c38101450 0x7f8c381018a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.138+0000 7f8c3d775700 1 -- 172.21.15.93:0/329909999 >> 172.21.15.93:0/329909999 conn(0x7f8c380fb880 msgr2=0x7f8c380fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:16.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.138+0000 7f8c3d775700 1 -- 172.21.15.93:0/329909999 shutdown_connections 2023-12-14T18:26:16.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.138+0000 7f8c3d775700 1 -- 172.21.15.93:0/329909999 wait complete. 2023-12-14T18:26:16.141 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c3d775700 1 Processor -- start 2023-12-14T18:26:16.142 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c3d775700 1 -- start start 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c3d775700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c38100260 0x7f8c38196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c3d775700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c38101450 0x7f8c381965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c3d775700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 0x7f8c3819d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c3d775700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3819dc00 con 0x7f8c38101de0 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c3d775700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3819dd40 con 0x7f8c38101450 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c3d775700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c3819ded0 con 0x7f8c38100260 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c377fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 0x7f8c3819d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c377fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 0x7f8c3819d630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46718/0 (socket says 172.21.15.93:46718) 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c377fe700 1 -- 172.21.15.93:0/3293606126 learned_addr learned my addr 172.21.15.93:0/3293606126 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c38100260 0x7f8c38196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c367fc700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c38101450 0x7f8c381965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c377fe700 1 -- 172.21.15.93:0/3293606126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c38100260 msgr2=0x7f8c38196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c377fe700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c38100260 0x7f8c38196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.143 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c377fe700 1 -- 172.21.15.93:0/3293606126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c38101450 msgr2=0x7f8c381965c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c377fe700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c38101450 0x7f8c381965c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.139+0000 7f8c377fe700 1 -- 172.21.15.93:0/3293606126 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30009d00 con 0x7f8c38101de0 2023-12-14T18:26:16.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.140+0000 7f8c377fe700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 0x7f8c3819d630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000b830 tx=0x7f8c30004c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:16.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.140+0000 7f8c1ffff700 1 -- 172.21.15.93:0/3293606126 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3000e040 con 0x7f8c38101de0 2023-12-14T18:26:16.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.140+0000 7f8c3d775700 1 -- 172.21.15.93:0/3293606126 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c3810bbe0 con 0x7f8c38101de0 2023-12-14T18:26:16.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.140+0000 7f8c1ffff700 1 -- 172.21.15.93:0/3293606126 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c3001db50 con 0x7f8c38101de0 2023-12-14T18:26:16.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.140+0000 7f8c1ffff700 1 -- 172.21.15.93:0/3293606126 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c3001bbd0 con 0x7f8c38101de0 2023-12-14T18:26:16.144 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.140+0000 7f8c3d775700 1 -- 172.21.15.93:0/3293606126 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c3810ba50 con 0x7f8c38101de0 2023-12-14T18:26:16.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.142+0000 7f8c1ffff700 1 -- 172.21.15.93:0/3293606126 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f8c3001dcc0 con 0x7f8c38101de0 2023-12-14T18:26:16.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.143+0000 7f8c1ffff700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8c200a1150 0x7f8c200a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.143+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8c200a1150 0x7f8c200a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.143+0000 7f8c1ffff700 1 -- 172.21.15.93:0/3293606126 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c300ceab0 con 0x7f8c38101de0 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.143+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8c200a1150 0x7f8c200a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c381012b0 tx=0x7f8c2800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.143+0000 7f8c49903b80 1 Processor -- start 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c49903b80 1 -- start start 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a158c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 0x563e29a16720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a16cb0 0x563e29a17120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c49903b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e298fe590 con 0x563e29a154b0 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c49903b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e2996d820 con 0x563e29a15e00 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c49903b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e29966cb0 con 0x563e29a16cb0 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c36ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a158c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c36ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a158c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46726/0 (socket says 172.21.15.93:46726) 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c36ffd700 1 -- 172.21.15.93:0/2479166525 learned_addr learned my addr 172.21.15.93:0/2479166525 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:16.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c367fc700 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 0x563e29a16720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c377fe700 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a16cb0 0x563e29a17120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c36ffd700 1 -- 172.21.15.93:0/2479166525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a16cb0 msgr2=0x563e29a17120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a16cb0 0x563e29a17120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c36ffd700 1 -- 172.21.15.93:0/2479166525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 msgr2=0x563e29a16720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 0x563e29a16720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c36ffd700 1 -- 172.21.15.93:0/2479166525 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e297e8360 con 0x563e29a154b0 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c377fe700 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a16cb0 0x563e29a17120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a158c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8c28010f70 tx=0x7f8c28013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0119f9009c22521 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c1d7fa700 1 -- 172.21.15.93:0/2479166525 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c28011db0 con 0x563e29a154b0 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c1d7fa700 1 -- 172.21.15.93:0/2479166525 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c28015070 con 0x563e29a154b0 2023-12-14T18:26:16.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.144+0000 7f8c1d7fa700 1 -- 172.21.15.93:0/2479166525 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2801c5b0 con 0x563e29a154b0 2023-12-14T18:26:16.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 -- 172.21.15.93:0/2479166525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 msgr2=0x563e29a158c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a158c0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8c28010f70 tx=0x7f8c28013000 comp rx=0 tx=0).stop 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 -- 172.21.15.93:0/2479166525 shutdown_connections 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a158c0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a16cb0 0x563e29a17120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 --2- 172.21.15.93:0/2479166525 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 0x563e29a16720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 -- 172.21.15.93:0/2479166525 >> 172.21.15.93:0/2479166525 conn(0x563e29a0ac50 msgr2=0x563e29a103e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 -- 172.21.15.93:0/2479166525 shutdown_connections 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 -- 172.21.15.93:0/2479166525 wait complete. 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.145+0000 7f8c49903b80 1 -- start start 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c49903b80 1 Processor -- start 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a1f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 0x563e29a1f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a16cb0 0x563e29a26680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c49903b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e29902f40 con 0x563e29a154b0 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c49903b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e29902be0 con 0x563e29a16cb0 2023-12-14T18:26:16.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c49903b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e297ea650 con 0x563e29a15e00 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c49903b80 10 client.0 _ll_register_callbacks cb 0x563e299268c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c36ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a1f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c36ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a1f100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46732/0 (socket says 172.21.15.93:46732) 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c36ffd700 1 -- 172.21.15.93:0/1354954674 learned_addr learned my addr 172.21.15.93:0/1354954674 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c367fc700 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 0x563e29a1f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c377fe700 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a16cb0 0x563e29a26680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c36ffd700 1 -- 172.21.15.93:0/1354954674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 msgr2=0x563e29a1f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 0x563e29a1f640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c36ffd700 1 -- 172.21.15.93:0/1354954674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a16cb0 msgr2=0x563e29a26680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a16cb0 0x563e29a26680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.146+0000 7f8c36ffd700 1 -- 172.21.15.93:0/1354954674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c28010c50 con 0x563e29a154b0 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a1f100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c28011e10 tx=0x7f8c28011e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c335b26867d422c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c1aff5700 1 -- 172.21.15.93:0/1354954674 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2802c070 con 0x563e29a154b0 2023-12-14T18:26:16.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c1aff5700 1 -- 172.21.15.93:0/1354954674 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c28017070 con 0x563e29a154b0 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c49903b80 1 -- 172.21.15.93:0/1354954674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563e29a08260 con 0x563e29a154b0 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c1aff5700 1 -- 172.21.15.93:0/1354954674 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c2801e630 con 0x563e29a154b0 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c49903b80 10 client.15435 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c49903b80 1 -- 172.21.15.93:0/1354954674 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563e2992d120 con 0x563e29a154b0 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c1aff5700 1 -- 172.21.15.93:0/1354954674 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563e29a08260 con 0x563e29a154b0 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c1aff5700 1 -- 172.21.15.93:0/1354954674 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c28026c80 con 0x563e29a154b0 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c49903b80 10 client.15435 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c49903b80 10 client.15435 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c49903b80 20 client.15435 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c49903b80 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563e29a2a260 0x563e29a29690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c49903b80 4 client.15435 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.147+0000 7f8c49903b80 1 -- 172.21.15.93:0/1354954674 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563e2995fc90 con 0x563e29a2a260 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.148+0000 7f8c367fc700 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563e29a2a260 0x563e29a29690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.148+0000 7f8c367fc700 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563e29a2a260 0x563e29a29690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d6c78667af777edd server_cookie=77fbcdad890afff8 in_seq=0 out_seq=0 2023-12-14T18:26:16.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.148+0000 7f8c1aff5700 10 client.15435 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c1aff5700 1 -- 172.21.15.93:0/1354954674 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563e2995fc90 con 0x563e29a2a260 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c1aff5700 10 client.15435 handle_command_reply: tid=0 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 client.15435 shutdown 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 -- 172.21.15.93:0/1354954674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 msgr2=0x563e29a1f100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a1f100 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8c28011e10 tx=0x7f8c28011e40 comp rx=0 tx=0).stop 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 -- 172.21.15.93:0/1354954674 shutdown_connections 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563e29a2a260 0x563e29a29690 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a1f100 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a16cb0 0x563e29a26680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c1aff5700 0 client.15435 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 --2- 172.21.15.93:0/1354954674 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 0x563e29a1f640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 -- 172.21.15.93:0/1354954674 >> 172.21.15.93:0/1354954674 conn(0x563e29a0ac50 msgr2=0x563e29a0eee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 -- 172.21.15.93:0/1354954674 shutdown_connections 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 1 -- 172.21.15.93:0/1354954674 wait complete. 2023-12-14T18:26:16.156 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.149+0000 7f8c49903b80 20 client.15435 trim_cache size 0 max 16384 2023-12-14T18:26:16.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.166+0000 7f8c49903b80 1 Processor -- start 2023-12-14T18:26:16.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.166+0000 7f8c49903b80 1 -- start start 2023-12-14T18:26:16.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.166+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a0ac50 0x563e29a19b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.166+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a154b0 0x563e29a26640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.166+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 0x563e29a1a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.166+0000 7f8c49903b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e29935a50 con 0x563e29a0ac50 2023-12-14T18:26:16.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.166+0000 7f8c49903b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e29910580 con 0x563e29a15e00 2023-12-14T18:26:16.173 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.166+0000 7f8c49903b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e29925290 con 0x563e29a154b0 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c36ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a0ac50 0x563e29a19b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c36ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a0ac50 0x563e29a19b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46738/0 (socket says 172.21.15.93:46738) 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c36ffd700 1 -- 172.21.15.93:0/921188250 learned_addr learned my addr 172.21.15.93:0/921188250 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c367fc700 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a154b0 0x563e29a26640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c377fe700 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 0x563e29a1a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c36ffd700 1 -- 172.21.15.93:0/921188250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a154b0 msgr2=0x563e29a26640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a154b0 0x563e29a26640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c36ffd700 1 -- 172.21.15.93:0/921188250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 msgr2=0x563e29a1a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 0x563e29a1a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c36ffd700 1 -- 172.21.15.93:0/921188250 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563e2992ce70 con 0x563e29a0ac50 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c367fc700 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a154b0 0x563e29a26640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c377fe700 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 0x563e29a1a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a0ac50 0x563e29a19b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8c28005d20 tx=0x7f8c2800b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1306a5c9e6c571c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:16.174 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c1dffb700 1 -- 172.21.15.93:0/921188250 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2802c030 con 0x563e29a0ac50 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c1dffb700 1 -- 172.21.15.93:0/921188250 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c28011a00 con 0x563e29a0ac50 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.167+0000 7f8c1dffb700 1 -- 172.21.15.93:0/921188250 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2801e890 con 0x563e29a0ac50 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 -- 172.21.15.93:0/921188250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a0ac50 msgr2=0x563e29a19b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a0ac50 0x563e29a19b50 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f8c28005d20 tx=0x7f8c2800b480 comp rx=0 tx=0).stop 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 -- 172.21.15.93:0/921188250 shutdown_connections 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a0ac50 0x563e29a19b50 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a15e00 0x563e29a1a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 --2- 172.21.15.93:0/921188250 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a154b0 0x563e29a26640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 -- 172.21.15.93:0/921188250 >> 172.21.15.93:0/921188250 conn(0x563e29a1e250 msgr2=0x563e29a29690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 -- 172.21.15.93:0/921188250 shutdown_connections 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 -- 172.21.15.93:0/921188250 wait complete. 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 -- start start 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 Processor -- start 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a0ac50 0x563e29a20da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a212e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.175 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 0x563e29a261f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e29a267a0 con 0x563e29a154b0 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e29a26910 con 0x563e29a0ac50 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563e29a7e0c0 con 0x563e29a15e00 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c49903b80 10 client.0 _ll_register_callbacks cb 0x7f8c2c00c370 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c367fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a212e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c367fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a212e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46752/0 (socket says 172.21.15.93:46752) 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c367fc700 1 -- 172.21.15.93:0/4005173634 learned_addr learned my addr 172.21.15.93:0/4005173634 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.168+0000 7f8c377fe700 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 0x563e29a261f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a0ac50 0x563e29a20da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c367fc700 1 -- 172.21.15.93:0/4005173634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 msgr2=0x563e29a261f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c367fc700 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 0x563e29a261f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c367fc700 1 -- 172.21.15.93:0/4005173634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a0ac50 msgr2=0x563e29a20da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c367fc700 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a0ac50 0x563e29a20da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c367fc700 1 -- 172.21.15.93:0/4005173634 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563e2992d120 con 0x563e29a154b0 2023-12-14T18:26:16.176 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c367fc700 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a212e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c004790 tx=0x7f8c2c01be60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75eefd02628f2b70 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c1aff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c1aff5700 1 -- 172.21.15.93:0/4005173634 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2c016040 con 0x563e29a154b0 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c1aff5700 1 -- 172.21.15.93:0/4005173634 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f8c2c01b5c0 con 0x563e29a154b0 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c49903b80 1 -- 172.21.15.93:0/4005173634 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c28015070 con 0x563e29a154b0 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c1aff5700 1 -- 172.21.15.93:0/4005173634 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c2c004a30 con 0x563e29a154b0 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c49903b80 10 client.15441 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c49903b80 1 -- 172.21.15.93:0/4005173634 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x563e29913950 con 0x563e29a154b0 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c1aff5700 1 -- 172.21.15.93:0/4005173634 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c28015070 con 0x563e29a154b0 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c1aff5700 1 -- 172.21.15.93:0/4005173634 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c2c023960 con 0x563e29a154b0 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c49903b80 10 client.15441 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c49903b80 10 client.15441 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.169+0000 7f8c49903b80 20 client.15441 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563e29a16cb0 0x563e29a7f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 4 client.15441 mds_command: new command op to 24461 tid=0[{"prefix": "status"}] 2023-12-14T18:26:16.177 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 -- 172.21.15.93:0/4005173634 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563e2995f9b0 con 0x563e29a16cb0 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563e29a16cb0 0x563e29a7f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c1aff5700 10 client.15441 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c36ffd700 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563e29a16cb0 0x563e29a7f6b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e1a96aaeee365e server_cookie=61ca079ea651930 in_seq=0 out_seq=0 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c1aff5700 1 -- 172.21.15.93:0/4005173634 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x563e2995f9b0 con 0x563e29a16cb0 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c1aff5700 10 client.15441 handle_command_reply: tid=0 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 client.15441 shutdown 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 -- 172.21.15.93:0/4005173634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 msgr2=0x563e29a212e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a212e0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f8c2c004790 tx=0x7f8c2c01be60 comp rx=0 tx=0).stop 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 -- 172.21.15.93:0/4005173634 shutdown_connections 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563e29a16cb0 0x563e29a7f6b0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563e29a15e00 0x563e29a261f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563e29a0ac50 0x563e29a20da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 --2- 172.21.15.93:0/4005173634 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563e29a154b0 0x563e29a212e0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.178 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c1aff5700 0 client.15441 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.170+0000 7f8c49903b80 1 -- 172.21.15.93:0/4005173634 >> 172.21.15.93:0/4005173634 conn(0x563e29a1e250 msgr2=0x563e29a19b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c49903b80 1 -- 172.21.15.93:0/4005173634 shutdown_connections 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c49903b80 1 -- 172.21.15.93:0/4005173634 wait complete. 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c49903b80 20 client.15441 trim_cache size 0 max 16384 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stdout: "cluster_fsid": "1c5ac00e-9aad-11ee-95a5-87774f69a715", 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stdout: "whoami": 0, 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stdout: "id": 24461, 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stdout: "want_state": "up:active", 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stdout: "state": "up:active", 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stdout: "fs_name": "cephfs", 2023-12-14T18:26:16.179 INFO:teuthology.orchestra.run.smithi093.stdout: "rank_uptime": 99.610840593000006, 2023-12-14T18:26:16.180 INFO:teuthology.orchestra.run.smithi093.stdout: "mdsmap_epoch": 24, 2023-12-14T18:26:16.180 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch": 86, 2023-12-14T18:26:16.180 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_epoch_barrier": 84, 2023-12-14T18:26:16.180 INFO:teuthology.orchestra.run.smithi093.stdout: "uptime": 105.083245967 2023-12-14T18:26:16.180 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:26:16.180 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 -- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8c200a1150 msgr2=0x7f8c200a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8c200a1150 0x7f8c200a3600 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f8c381012b0 tx=0x7f8c2800b410 comp rx=0 tx=0).stop 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 -- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 msgr2=0x7f8c3819d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 0x7f8c3819d630 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f8c3000b830 tx=0x7f8c30004c80 comp rx=0 tx=0).stop 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 -- 172.21.15.93:0/3293606126 shutdown_connections 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f8c200a1150 0x7f8c200a3600 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c38101de0 0x7f8c3819d630 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c38100260 0x7f8c38196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 --2- 172.21.15.93:0/3293606126 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c38101450 0x7f8c381965c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 -- 172.21.15.93:0/3293606126 >> 172.21.15.93:0/3293606126 conn(0x7f8c380fb880 msgr2=0x7f8c380fda30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 -- 172.21.15.93:0/3293606126 shutdown_connections 2023-12-14T18:26:16.181 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:16.171+0000 7f8c3d775700 1 -- 172.21.15.93:0/3293606126 wait complete. 2023-12-14T18:26:16.185 INFO:teuthology.run_tasks:Running task kclient... 2023-12-14T18:26:16.197 INFO:tasks.kclient:Mounting kernel clients... 2023-12-14T18:26:16.198 INFO:tasks.kclient:config is {'client.0': {}} 2023-12-14T18:26:16.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:16.198 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-12-14T18:26:16.215 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2023-12-14T18:26:16.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:16.216 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2023-12-14T18:26:16.279 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:16 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/747423473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:16.279 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:16 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4278512348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:16.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:16.290 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2023-12-14T18:26:16.398 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=crc', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2023-12-14T18:26:16.399 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-14T18:26:16.399 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-14T18:26:16.399 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph fs ls 2023-12-14T18:26:16.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:16 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/747423473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:16.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:16 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4278512348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:16.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:16 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/747423473' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:26:16.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:16 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4278512348' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:26:16.712 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:26:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:17 smithi160 ceph-mon[43840]: pgmap v276: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-12-14T18:26:17.529 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:17 smithi093 ceph-mon[42070]: pgmap v276: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-12-14T18:26:17.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:17 smithi097 ceph-mon[43486]: pgmap v276: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 426 B/s wr, 6 op/s 2023-12-14T18:26:17.728 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.726+0000 7fbca7d3c700 1 -- 172.21.15.93:0/3706060357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca00ff470 msgr2=0x7fbca00ff840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.726+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/3706060357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca00ff470 0x7fbca00ff840 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fbc900099c0 tx=0x7fbc90009cd0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.726+0000 7fbca7d3c700 1 -- 172.21.15.93:0/3706060357 shutdown_connections 2023-12-14T18:26:17.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.726+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/3706060357 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca00ff470 0x7fbca00ff840 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.726+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/3706060357 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca0104510 0x7fbca010c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.726+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/3706060357 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbca00ffe10 0x7fbca0103de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.726+0000 7fbca7d3c700 1 -- 172.21.15.93:0/3706060357 >> 172.21.15.93:0/3706060357 conn(0x7fbca00770d0 msgr2=0x7fbca00774d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:17.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 -- 172.21.15.93:0/3706060357 shutdown_connections 2023-12-14T18:26:17.729 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 -- 172.21.15.93:0/3706060357 wait complete. 2023-12-14T18:26:17.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 Processor -- start 2023-12-14T18:26:17.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 -- start start 2023-12-14T18:26:17.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca00ff470 0x7fbca0195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.731 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbca00ffe10 0x7fbca01957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca0104510 0x7fbca0199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca019a250 con 0x7fbca0104510 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca019a3c0 con 0x7fbca00ffe10 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca7d3c700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca019a580 con 0x7fbca00ff470 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.727+0000 7fbca62d9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca0104510 0x7fbca0199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca62d9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca0104510 0x7fbca0199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46766/0 (socket says 172.21.15.93:46766) 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca62d9700 1 -- 172.21.15.93:0/1747120382 learned_addr learned my addr 172.21.15.93:0/1747120382 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca5ad8700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca00ff470 0x7fbca0195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca52d7700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbca00ffe10 0x7fbca01957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca62d9700 1 -- 172.21.15.93:0/1747120382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca00ff470 msgr2=0x7fbca0195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.732 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca62d9700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca00ff470 0x7fbca0195290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca62d9700 1 -- 172.21.15.93:0/1747120382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbca00ffe10 msgr2=0x7fbca01957d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca62d9700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbca00ffe10 0x7fbca01957d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca62d9700 1 -- 172.21.15.93:0/1747120382 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900095f0 con 0x7fbca0104510 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca52d7700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbca00ffe10 0x7fbca01957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca62d9700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca0104510 0x7fbca0199c40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc98007ee0 tx=0x7fbc9800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbc96ffd700 1 -- 172.21.15.93:0/1747120382 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98017070 con 0x7fbca0104510 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca010f250 con 0x7fbca0104510 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbc96ffd700 1 -- 172.21.15.93:0/1747120382 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbc9800f040 con 0x7fbca0104510 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbc96ffd700 1 -- 172.21.15.93:0/1747120382 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc98005740 con 0x7fbca0104510 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.728+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca010f0c0 con 0x7fbca0104510 2023-12-14T18:26:17.733 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.729+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca010f3e0 con 0x7fbca0104510 2023-12-14T18:26:17.734 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.730+0000 7fbc96ffd700 1 -- 172.21.15.93:0/1747120382 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7fbc9800dc60 con 0x7fbca0104510 2023-12-14T18:26:17.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.731+0000 7fbc96ffd700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbc8c0a1150 0x7fbc8c0a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:17.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.731+0000 7fbca5ad8700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbc8c0a1150 0x7fbc8c0a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:17.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.731+0000 7fbc96ffd700 1 -- 172.21.15.93:0/1747120382 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fbc980cc3d0 con 0x7fbca0104510 2023-12-14T18:26:17.735 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.731+0000 7fbca5ad8700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbc8c0a1150 0x7fbc8c0a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009990 tx=0x7fbc90003680 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:17.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.735+0000 7fbc96ffd700 1 -- 172.21.15.93:0/1747120382 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbc98095810 con 0x7fbca0104510 2023-12-14T18:26:17.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.967+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fbca0066ea0 con 0x7fbca0104510 2023-12-14T18:26:17.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.967+0000 7fbc96ffd700 1 -- 172.21.15.93:0/1747120382 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+88 (secure 0 0 0) 0x7fbc98098e30 con 0x7fbca0104510 2023-12-14T18:26:17.969 INFO:teuthology.orchestra.run.smithi093.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2023-12-14T18:26:17.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbc8c0a1150 msgr2=0x7fbc8c0a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbc8c0a1150 0x7fbc8c0a3600 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009990 tx=0x7fbc90003680 comp rx=0 tx=0).stop 2023-12-14T18:26:17.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca0104510 msgr2=0x7fbca0199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:17.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca0104510 0x7fbca0199c40 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fbc98007ee0 tx=0x7fbc9800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 shutdown_connections 2023-12-14T18:26:17.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fbc8c0a1150 0x7fbc8c0a3600 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbca0104510 0x7fbca0199c40 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbca00ff470 0x7fbca0195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 --2- 172.21.15.93:0/1747120382 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbca00ffe10 0x7fbca01957d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:17.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 >> 172.21.15.93:0/1747120382 conn(0x7fbca00770d0 msgr2=0x7fbca00fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:17.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 shutdown_connections 2023-12-14T18:26:17.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:17.969+0000 7fbca7d3c700 1 -- 172.21.15.93:0/1747120382 wait complete. 2023-12-14T18:26:18.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:18 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1747120382' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:26:18.385 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-14T18:26:18.385 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-14T18:26:18.385 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi093.front.sepia.ceph.com 2023-12-14T18:26:18.385 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-14T18:26:18.385 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-14T18:26:18.385 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-14T18:26:18.385 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-14T18:26:18.385 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-14T18:26:18.385 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-14T18:26:18.385 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:18.386 DEBUG:teuthology.orchestra.run.smithi093:> ip addr 2023-12-14T18:26:18.404 INFO:teuthology.orchestra.run.smithi093.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-14T18:26:18.404 INFO:teuthology.orchestra.run.smithi093.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout: inet 127.0.0.1/8 scope host lo 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 ::1/128 scope host 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1c brd ff:ff:ff:ff:ff:ff 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f0 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:d8:da:1d brd ff:ff:ff:ff:ff:ff 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout: altname enp7s0f1 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-12-14T18:26:18.405 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:34 brd ff:ff:ff:ff:ff:ff 2023-12-14T18:26:18.406 INFO:teuthology.orchestra.run.smithi093.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-14T18:26:18.406 INFO:teuthology.orchestra.run.smithi093.stdout: link/ether 0c:c4:7a:8f:d6:35 brd ff:ff:ff:ff:ff:ff 2023-12-14T18:26:18.406 INFO:teuthology.orchestra.run.smithi093.stdout: inet 172.21.15.93/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-12-14T18:26:18.406 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft 41973sec preferred_lft 41973sec 2023-12-14T18:26:18.406 INFO:teuthology.orchestra.run.smithi093.stdout: inet6 fe80::1355:f7c1:e067:88d9/64 scope link noprefixroute 2023-12-14T18:26:18.406 INFO:teuthology.orchestra.run.smithi093.stdout: valid_lft forever preferred_lft forever 2023-12-14T18:26:18.406 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-14T18:26:18.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:18.406 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:26:18.406 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:26:18.407 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add name ceph-brx type bridge 2023-12-14T18:26:18.407 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr flush dev ceph-brx 2023-12-14T18:26:18.407 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set ceph-brx up 2023-12-14T18:26:18.407 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-14T18:26:18.407 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:26:18.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:18 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1747120382' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:26:18.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:18.526 DEBUG:teuthology.orchestra.run.smithi093:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-14T18:26:18.553 INFO:teuthology.orchestra.run.smithi093.stdout:1 2023-12-14T18:26:18.555 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:18.555 DEBUG:teuthology.orchestra.run.smithi093:> route 2023-12-14T18:26:18.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:18 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1747120382' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2023-12-14T18:26:18.615 INFO:teuthology.orchestra.run.smithi093.stdout:Kernel IP routing table 2023-12-14T18:26:18.615 INFO:teuthology.orchestra.run.smithi093.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-14T18:26:18.615 INFO:teuthology.orchestra.run.smithi093.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-12-14T18:26:18.615 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-12-14T18:26:18.615 INFO:teuthology.orchestra.run.smithi093.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-14T18:26:18.616 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:18.616 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:26:18.616 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:26:18.616 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-12-14T18:26:18.616 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-12-14T18:26:18.616 DEBUG:teuthology.orchestra.run.smithi093:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-12-14T18:26:18.616 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:26:18.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:18.722 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-12-14T18:26:18.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:18.739 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list-id 2023-12-14T18:26:18.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:18.795 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:26:18.796 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:26:18.796 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-14T18:26:18.796 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-14T18:26:18.796 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:26:18.890 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-14T18:26:18.890 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:18.890 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:26:18.890 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:26:18.890 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-14T18:26:18.890 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-14T18:26:18.890 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-14T18:26:18.891 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-14T18:26:18.891 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-14T18:26:18.891 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:26:19.108 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:19.108 DEBUG:teuthology.orchestra.run.smithi093:> (cd / && exec bash -c ' 2023-12-14T18:26:19.108 DEBUG:teuthology.orchestra.run.smithi093:> set -e 2023-12-14T18:26:19.108 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set brx.0 up 2023-12-14T18:26:19.108 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link set dev brx.0 master ceph-brx 2023-12-14T18:26:19.109 DEBUG:teuthology.orchestra.run.smithi093:> ') 2023-12-14T18:26:19.199 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:19 smithi093 ceph-mon[42070]: pgmap v277: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:26:19.206 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-14T18:26:19.207 DEBUG:teuthology.orchestra.run.smithi093:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-14T18:26:19.223 INFO:teuthology.orchestra.run.smithi093.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-14T18:26:19.223 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-14T18:26:19.224 DEBUG:teuthology.orchestra.run.smithi093:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-14T18:26:19.280 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2023-12-14T18:26:19.280 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2023-12-14T18:26:19.280 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/_nogroup/sv_1/17dc08c9-cb20-4697-9d38-a6077c301cb3 2023-12-14T18:26:19.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:19.280 DEBUG:teuthology.orchestra.run.smithi093:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/_nogroup/sv_1/17dc08c9-cb20-4697-9d38-a6077c301cb3 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2023-12-14T18:26:19.366 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2023-12-14T18:26:19.366 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2023-12-14T18:26:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:19 smithi160 ceph-mon[43840]: pgmap v277: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:26:19.504 INFO:teuthology.orchestra.run.smithi093.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=crc,wsync,nofallback 2023-12-14T18:26:19.504 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync". 2023-12-14T18:26:19.504 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: resolved to: "172.21.15.93:3300,172.21.15.97:3300,172.21.15.160:3300" 2023-12-14T18:26:19.504 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: trying mount with new device syntax: 0@1c5ac00e-9aad-11ee-95a5-87774f69a715.cephfs=/volumes/_nogroup/sv_1/17dc08c9-cb20-4697-9d38-a6077c301cb3 2023-12-14T18:26:19.504 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mon_addr=172.21.15.93:3300/172.21.15.97:3300/172.21.15.160:3300" will pass to kernel 2023-12-14T18:26:19.504 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: v2 mount support check returned 2 2023-12-14T18:26:19.504 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: kernel does not support v2 syntax 2023-12-14T18:26:19.504 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.93:3300,172.21.15.97:3300,172.21.15.160:3300:/volumes/_nogroup/sv_1/17dc08c9-cb20-4697-9d38-a6077c301cb3 2023-12-14T18:26:19.504 INFO:teuthology.orchestra.run.smithi093.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=crc,wsync,name=0,key=0,mds_namespace=cephfs,fsid=1c5ac00e-9aad-11ee-95a5-87774f69a715" will pass to kernel 2023-12-14T18:26:19.506 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-14T18:26:19.506 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:19.507 DEBUG:teuthology.orchestra.run.smithi093:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-14T18:26:19.535 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:19.535 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-14T18:26:19.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:19 smithi097 ceph-mon[43486]: pgmap v277: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 426 B/s wr, 7 op/s 2023-12-14T18:26:19.609 INFO:teuthology.orchestra.run.smithi093.stdout:client15450 2023-12-14T18:26:19.609 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:26:19.609 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:26:19.668 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:19.668 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-14T18:26:19.741 INFO:teuthology.orchestra.run.smithi093.stdout:client15450 2023-12-14T18:26:19.741 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:19.742 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-14T18:26:19.814 INFO:teuthology.orchestra.run.smithi093.stdout:client15450 2023-12-14T18:26:19.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:26:19.815 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715.client15450/status') 2023-12-14T18:26:19.895 INFO:teuthology.orchestra.run.smithi093.stdout:instance: client.15450 (3)192.168.0.1:0/70157731 2023-12-14T18:26:19.895 INFO:teuthology.orchestra.run.smithi093.stdout:blocklisted: false 2023-12-14T18:26:19.896 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records in 2023-12-14T18:26:19.896 INFO:teuthology.orchestra.run.smithi093.stderr:0+1 records out 2023-12-14T18:26:19.896 INFO:teuthology.orchestra.run.smithi093.stderr:68 bytes copied, 2.372e-05 s, 2.9 MB/s 2023-12-14T18:26:19.897 INFO:teuthology.run_tasks:Running task check-counter... 2023-12-14T18:26:19.908 INFO:teuthology.run_tasks:Running task exec... 2023-12-14T18:26:19.947 INFO:teuthology.task.exec:Executing custom commands... 2023-12-14T18:26:19.947 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:26:19.947 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2023-12-14T18:26:20.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.067+0000 7f1b97166700 1 -- 172.21.15.93:0/2279842718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 msgr2=0x7f1b90100670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:20.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.067+0000 7f1b97166700 1 --2- 172.21.15.93:0/2279842718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 0x7f1b90100670 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f1b84009990 tx=0x7f1b84009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.067+0000 7f1b97166700 1 -- 172.21.15.93:0/2279842718 shutdown_connections 2023-12-14T18:26:20.071 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.067+0000 7f1b97166700 1 --2- 172.21.15.93:0/2279842718 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 0x7f1b90100670 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.067+0000 7f1b97166700 1 --2- 172.21.15.93:0/2279842718 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b90101ea0 0x7f1b90108740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.067+0000 7f1b97166700 1 --2- 172.21.15.93:0/2279842718 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b901014f0 0x7f1b90101960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.067+0000 7f1b97166700 1 -- 172.21.15.93:0/2279842718 >> 172.21.15.93:0/2279842718 conn(0x7f1b900fb880 msgr2=0x7f1b900fdcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.067+0000 7f1b97166700 1 -- 172.21.15.93:0/2279842718 shutdown_connections 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.067+0000 7f1b97166700 1 -- 172.21.15.93:0/2279842718 wait complete. 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b97166700 1 Processor -- start 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b97166700 1 -- start start 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b97166700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 0x7f1b90195f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b97166700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b901014f0 0x7f1b90196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b97166700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b90101ea0 0x7f1b9019d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b97166700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9019da90 con 0x7f1b90100260 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b97166700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9019dbd0 con 0x7f1b90101ea0 2023-12-14T18:26:20.072 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b97166700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9019dd60 con 0x7f1b901014f0 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b94f02700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 0x7f1b90195f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b94f02700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 0x7f1b90195f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46796/0 (socket says 172.21.15.93:46796) 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b94f02700 1 -- 172.21.15.93:0/4197811593 learned_addr learned my addr 172.21.15.93:0/4197811593 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b8ffff700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b901014f0 0x7f1b90196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b95703700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b90101ea0 0x7f1b9019d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b94f02700 1 -- 172.21.15.93:0/4197811593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b901014f0 msgr2=0x7f1b90196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b94f02700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b901014f0 0x7f1b90196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b94f02700 1 -- 172.21.15.93:0/4197811593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b90101ea0 msgr2=0x7f1b9019d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b94f02700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b90101ea0 0x7f1b9019d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b94f02700 1 -- 172.21.15.93:0/4197811593 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b840095f0 con 0x7f1b90100260 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b95703700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b90101ea0 0x7f1b9019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:20.073 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.068+0000 7f1b94f02700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 0x7f1b90195f10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1b84000c00 tx=0x7f1b8400f710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:20.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.069+0000 7f1b8dffb700 1 -- 172.21.15.93:0/4197811593 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8401d070 con 0x7f1b90100260 2023-12-14T18:26:20.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.069+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9010bc00 con 0x7f1b90100260 2023-12-14T18:26:20.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.069+0000 7f1b8dffb700 1 -- 172.21.15.93:0/4197811593 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1b8400fb50 con 0x7f1b90100260 2023-12-14T18:26:20.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.069+0000 7f1b8dffb700 1 -- 172.21.15.93:0/4197811593 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b84017650 con 0x7f1b90100260 2023-12-14T18:26:20.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.069+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9010ba70 con 0x7f1b90100260 2023-12-14T18:26:20.074 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.070+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b90107800 con 0x7f1b90100260 2023-12-14T18:26:20.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.071+0000 7f1b8dffb700 1 -- 172.21.15.93:0/4197811593 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 18) v1 ==== 135715+0+0 (secure 0 0 0) 0x7f1b8400fcc0 con 0x7f1b90100260 2023-12-14T18:26:20.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.072+0000 7f1b8dffb700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1b800a1150 0x7f1b800a3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:20.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.072+0000 7f1b8ffff700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1b800a1150 0x7f1b800a3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:20.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.072+0000 7f1b8dffb700 1 -- 172.21.15.93:0/4197811593 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f1b840ce2e0 con 0x7f1b90100260 2023-12-14T18:26:20.075 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.072+0000 7f1b8ffff700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1b800a1150 0x7f1b800a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c005950 tx=0x7f1b7c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:20.077 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.075+0000 7f1b8dffb700 1 -- 172.21.15.93:0/4197811593 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1b840977b0 con 0x7f1b90100260 2023-12-14T18:26:20.325 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:20.324+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f1b90061700 con 0x7f1b90100260 2023-12-14T18:26:21.134 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.132+0000 7f1b8dffb700 1 -- 172.21.15.93:0/4197811593 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f1b84097b80 con 0x7f1b90100260 2023-12-14T18:26:21.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.134+0000 7f1b8dffb700 1 -- 172.21.15.93:0/4197811593 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7f1b8405c0c0 con 0x7f1b90100260 2023-12-14T18:26:21.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1b800a1150 msgr2=0x7f1b800a3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1b800a1150 0x7f1b800a3600 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f1b7c005950 tx=0x7f1b7c00b410 comp rx=0 tx=0).stop 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 msgr2=0x7f1b90195f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 0x7f1b90195f10 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f1b84000c00 tx=0x7f1b8400f710 comp rx=0 tx=0).stop 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 shutdown_connections 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f1b800a1150 0x7f1b800a3600 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1b90100260 0x7f1b90195f10 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1b90101ea0 0x7f1b9019d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 --2- 172.21.15.93:0/4197811593 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1b901014f0 0x7f1b90196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 >> 172.21.15.93:0/4197811593 conn(0x7f1b900fb880 msgr2=0x7f1b90104940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:21.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 shutdown_connections 2023-12-14T18:26:21.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.136+0000 7f1b97166700 1 -- 172.21.15.93:0/4197811593 wait complete. 2023-12-14T18:26:21.152 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2023-12-14T18:26:21.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.282+0000 7f19fde51700 1 -- 172.21.15.93:0/3953109764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80ffe60 msgr2=0x7f19f81002d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.287 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.282+0000 7f19fde51700 1 --2- 172.21.15.93:0/3953109764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80ffe60 0x7f19f81002d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f19f000b0a0 tx=0x7f19f000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.282+0000 7f19fde51700 1 -- 172.21.15.93:0/3953109764 shutdown_connections 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.282+0000 7f19fde51700 1 --2- 172.21.15.93:0/3953109764 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80ffe60 0x7f19f81002d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.282+0000 7f19fde51700 1 --2- 172.21.15.93:0/3953109764 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f80feb60 0x7f19f80fef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.282+0000 7f19fde51700 1 --2- 172.21.15.93:0/3953109764 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19f80ff4b0 0x7f19f80ff920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.282+0000 7f19fde51700 1 -- 172.21.15.93:0/3953109764 >> 172.21.15.93:0/3953109764 conn(0x7f19f80fa7b0 msgr2=0x7f19f80fcbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.282+0000 7f19fde51700 1 -- 172.21.15.93:0/3953109764 shutdown_connections 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.282+0000 7f19fde51700 1 -- 172.21.15.93:0/3953109764 wait complete. 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19fde51700 1 Processor -- start 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19fde51700 1 -- start start 2023-12-14T18:26:21.288 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19fde51700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80feb60 0x7f19f81982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19fde51700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f80ff4b0 0x7f19f81987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19fde51700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19f80ffe60 0x7f19f819f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19fde51700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f819fde0 con 0x7f19f80feb60 2023-12-14T18:26:21.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19fde51700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f819ff20 con 0x7f19f80ffe60 2023-12-14T18:26:21.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19fde51700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f81a00b0 con 0x7f19f80ff4b0 2023-12-14T18:26:21.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19f77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80feb60 0x7f19f81982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19f77fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80feb60 0x7f19f81982b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46820/0 (socket says 172.21.15.93:46820) 2023-12-14T18:26:21.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.283+0000 7f19f77fe700 1 -- 172.21.15.93:0/1358789782 learned_addr learned my addr 172.21.15.93:0/1358789782 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:21.289 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f6ffd700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f80ff4b0 0x7f19f81987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f7fff700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19f80ffe60 0x7f19f819f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f77fe700 1 -- 172.21.15.93:0/1358789782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f80ff4b0 msgr2=0x7f19f81987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f77fe700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f80ff4b0 0x7f19f81987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f77fe700 1 -- 172.21.15.93:0/1358789782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19f80ffe60 msgr2=0x7f19f819f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f77fe700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19f80ffe60 0x7f19f819f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f77fe700 1 -- 172.21.15.93:0/1358789782 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f0009d00 con 0x7f19f80feb60 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f77fe700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80feb60 0x7f19f81982b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f19e800e9a0 tx=0x7f19e800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f4ff9700 1 -- 172.21.15.93:0/1358789782 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8009790 con 0x7f19f80feb60 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f4ff9700 1 -- 172.21.15.93:0/1358789782 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f19e8010430 con 0x7f19f80feb60 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f810ddd0 con 0x7f19f80feb60 2023-12-14T18:26:21.290 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19f4ff9700 1 -- 172.21.15.93:0/1358789782 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e8018680 con 0x7f19f80feb60 2023-12-14T18:26:21.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.284+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f810dc40 con 0x7f19f80feb60 2023-12-14T18:26:21.291 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.285+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f8109940 con 0x7f19f80feb60 2023-12-14T18:26:21.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.287+0000 7f19f4ff9700 1 -- 172.21.15.93:0/1358789782 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f19e80105a0 con 0x7f19f80feb60 2023-12-14T18:26:21.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.288+0000 7f19f4ff9700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19e00a11a0 0x7f19e00a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.288+0000 7f19f6ffd700 1 -- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19e00a11a0 msgr2=0x7f19e00a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3802499407 2023-12-14T18:26:21.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.288+0000 7f19f4ff9700 1 -- 172.21.15.93:0/1358789782 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f19e8014070 con 0x7f19f80feb60 2023-12-14T18:26:21.292 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.288+0000 7f19f6ffd700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19e00a11a0 0x7f19e00a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:26:21.294 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.292+0000 7f19f4ff9700 1 -- 172.21.15.93:0/1358789782 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f19e8051730 con 0x7f19f80feb60 2023-12-14T18:26:21.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.489+0000 7f19f6ffd700 1 -- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19e00a11a0 msgr2=0x7f19e00a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3802499407 2023-12-14T18:26:21.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.489+0000 7f19f6ffd700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19e00a11a0 0x7f19e00a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:26:21.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.490+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f19f802cfe0 con 0x7f19f80feb60 2023-12-14T18:26:21.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:21 smithi160 ceph-mon[43840]: pgmap v278: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2023-12-14T18:26:21.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:21 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4197811593' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-14T18:26:21.496 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: ignoring --setuser ceph since I am not root 2023-12-14T18:26:21.497 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: ignoring --setgroup ceph since I am not root 2023-12-14T18:26:21.497 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:21 smithi160 ceph-mgr[45713]: -- 172.21.15.160:0/1008092629 <== mon.1 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561205921520 con 0x5612058f7000 2023-12-14T18:26:21.497 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.495+0000 7f19f4ff9700 1 -- 172.21.15.93:0/1358789782 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7f19e8099d10 con 0x7f19f80feb60 2023-12-14T18:26:21.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19e00a11a0 msgr2=0x7f19e00a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:26:21.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19e00a11a0 0x7f19e00a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80feb60 msgr2=0x7f19f81982b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80feb60 0x7f19f81982b0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f19e800e9a0 tx=0x7f19e800ecb0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 shutdown_connections 2023-12-14T18:26:21.500 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f19e00a11a0 0x7f19e00a3650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f19f80feb60 0x7f19f81982b0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f19f80ffe60 0x7f19f819f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 --2- 172.21.15.93:0/1358789782 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f19f80ff4b0 0x7f19f81987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 >> 172.21.15.93:0/1358789782 conn(0x7f19f80fa7b0 msgr2=0x7f19f80fc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:21.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 shutdown_connections 2023-12-14T18:26:21.501 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.497+0000 7f19fde51700 1 -- 172.21.15.93:0/1358789782 wait complete. 2023-12-14T18:26:21.502 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:21 smithi097 ceph-mon[43486]: pgmap v278: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2023-12-14T18:26:21.502 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:21 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4197811593' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-14T18:26:21.502 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: ignoring --setuser ceph since I am not root 2023-12-14T18:26:21.502 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: ignoring --setgroup ceph since I am not root 2023-12-14T18:26:21.502 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:21 smithi097 ceph-mgr[44957]: -- 172.21.15.97:0/4063076258 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5609770b1520 con 0x560977086800 2023-12-14T18:26:21.508 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:21 smithi093 ceph-mon[42070]: pgmap v278: 129 pgs: 129 active+clean; 640 KiB data, 79 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 0 B/s wr, 5 op/s 2023-12-14T18:26:21.508 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:21 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4197811593' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2023-12-14T18:26:21.509 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: ignoring --setuser ceph since I am not root 2023-12-14T18:26:21.509 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: ignoring --setgroup ceph since I am not root 2023-12-14T18:26:21.509 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:21 smithi093 ceph-mgr[42347]: -- 172.21.15.93:0/2256097696 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x561bcc93b520 con 0x561bcc910800 2023-12-14T18:26:21.509 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:21.506+0000 7fc63c20f200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:26:21.513 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2023-12-14T18:26:21.645 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.638+0000 7fcf24163700 1 -- 172.21.15.93:0/582887306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c0ffd90 msgr2=0x7fcf1c107660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.638+0000 7fcf24163700 1 --2- 172.21.15.93:0/582887306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c0ffd90 0x7fcf1c107660 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400b0a0 tx=0x7fcf1400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 -- 172.21.15.93:0/582887306 shutdown_connections 2023-12-14T18:26:21.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 --2- 172.21.15.93:0/582887306 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c0ffd90 0x7fcf1c107660 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 --2- 172.21.15.93:0/582887306 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf1c1006d0 0x7fcf1c100ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 --2- 172.21.15.93:0/582887306 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1c0ff400 0x7fcf1c0ff850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 -- 172.21.15.93:0/582887306 >> 172.21.15.93:0/582887306 conn(0x7fcf1c0f9ea0 msgr2=0x7fcf1c0fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:21.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 -- 172.21.15.93:0/582887306 shutdown_connections 2023-12-14T18:26:21.646 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 -- 172.21.15.93:0/582887306 wait complete. 2023-12-14T18:26:21.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 Processor -- start 2023-12-14T18:26:21.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 -- start start 2023-12-14T18:26:21.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf1c0ff400 0x7fcf1c0fee80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1c0ffd90 0x7fcf1c19ada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c1006d0 0x7fcf1c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c19dbc0 con 0x7fcf1c1006d0 2023-12-14T18:26:21.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c19dd30 con 0x7fcf1c0ffd90 2023-12-14T18:26:21.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.639+0000 7fcf24163700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf1c19def0 con 0x7fcf1c0ff400 2023-12-14T18:26:21.647 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf22700700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c1006d0 0x7fcf1c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf21eff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf1c0ff400 0x7fcf1c0fee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf22700700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c1006d0 0x7fcf1c19d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42664/0 (socket says 172.21.15.93:42664) 2023-12-14T18:26:21.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf22700700 1 -- 172.21.15.93:0/2404951887 learned_addr learned my addr 172.21.15.93:0/2404951887 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:21.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf216fe700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1c0ffd90 0x7fcf1c19ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:21.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf22700700 1 -- 172.21.15.93:0/2404951887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf1c0ff400 msgr2=0x7fcf1c0fee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf22700700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf1c0ff400 0x7fcf1c0fee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf22700700 1 -- 172.21.15.93:0/2404951887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1c0ffd90 msgr2=0x7fcf1c19ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.648 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf22700700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1c0ffd90 0x7fcf1c19ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf22700700 1 -- 172.21.15.93:0/2404951887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf14009d00 con 0x7fcf1c1006d0 2023-12-14T18:26:21.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf21eff700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf1c0ff400 0x7fcf1c0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:21.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf22700700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c1006d0 0x7fcf1c19d3f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400b780 tx=0x7fcf1400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:21.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf12ffd700 1 -- 172.21.15.93:0/2404951887 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf1400e040 con 0x7fcf1c1006d0 2023-12-14T18:26:21.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf12ffd700 1 -- 172.21.15.93:0/2404951887 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcf140094a0 con 0x7fcf1c1006d0 2023-12-14T18:26:21.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf1c10aaa0 con 0x7fcf1c1006d0 2023-12-14T18:26:21.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf12ffd700 1 -- 172.21.15.93:0/2404951887 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf14005390 con 0x7fcf1c1006d0 2023-12-14T18:26:21.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.640+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf1c10a910 con 0x7fcf1c1006d0 2023-12-14T18:26:21.649 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.641+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf1c106760 con 0x7fcf1c1006d0 2023-12-14T18:26:21.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.643+0000 7fcf12ffd700 1 -- 172.21.15.93:0/2404951887 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7fcf1401d470 con 0x7fcf1c1006d0 2023-12-14T18:26:21.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.643+0000 7fcf12ffd700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcf080a1220 0x7fcf080a36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.643+0000 7fcf21eff700 1 -- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcf080a1220 msgr2=0x7fcf080a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3802499407 2023-12-14T18:26:21.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.643+0000 7fcf12ffd700 1 -- 172.21.15.93:0/2404951887 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7fcf140ce510 con 0x7fcf1c1006d0 2023-12-14T18:26:21.650 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.643+0000 7fcf21eff700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcf080a1220 0x7fcf080a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:26:21.651 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.647+0000 7fcf12ffd700 1 -- 172.21.15.93:0/2404951887 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fcf140979e0 con 0x7fcf1c1006d0 2023-12-14T18:26:21.776 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:21.500+0000 7f34af70e200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:26:21.776 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:21.601+0000 7f34af70e200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:26:21.776 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:21.689+0000 7f34af70e200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:26:21.777 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:21.501+0000 7f5a10709200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-14T18:26:21.777 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:21.602+0000 7f5a10709200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:26:21.777 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:21.691+0000 7f5a10709200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:26:21.799 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:21.615+0000 7fc63c20f200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-14T18:26:21.799 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:21.709+0000 7fc63c20f200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-14T18:26:21.799 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:21.797+0000 7fc63c20f200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:26:21.845 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.844+0000 7fcf21eff700 1 -- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcf080a1220 msgr2=0x7fcf080a36d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3802499407 2023-12-14T18:26:21.846 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.844+0000 7fcf21eff700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcf080a1220 0x7fcf080a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:26:21.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.848+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fcf1c02cfe0 con 0x7fcf1c1006d0 2023-12-14T18:26:21.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.853+0000 7fcf12ffd700 1 -- 172.21.15.93:0/2404951887 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7fcf1409b000 con 0x7fcf1c1006d0 2023-12-14T18:26:21.857 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcf080a1220 msgr2=0x7fcf080a36d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcf080a1220 0x7fcf080a36d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c1006d0 msgr2=0x7fcf1c19d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c1006d0 0x7fcf1c19d3f0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7fcf1400b780 tx=0x7fcf1400bd60 comp rx=0 tx=0).stop 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 shutdown_connections 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7fcf080a1220 0x7fcf080a36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcf1c1006d0 0x7fcf1c19d3f0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcf1c0ff400 0x7fcf1c0fee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 --2- 172.21.15.93:0/2404951887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcf1c0ffd90 0x7fcf1c19ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 >> 172.21.15.93:0/2404951887 conn(0x7fcf1c0f9ea0 msgr2=0x7fcf1c1038a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 shutdown_connections 2023-12-14T18:26:21.858 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.855+0000 7fcf24163700 1 -- 172.21.15.93:0/2404951887 wait complete. 2023-12-14T18:26:21.870 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1M' 2023-12-14T18:26:21.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.994+0000 7f6e5860e700 1 -- 172.21.15.93:0/2230668411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e50105fa0 msgr2=0x7f6e50108380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:21.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.994+0000 7f6e5860e700 1 --2- 172.21.15.93:0/2230668411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e50105fa0 0x7f6e50108380 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6e4800b0a0 tx=0x7f6e4800b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 -- 172.21.15.93:0/2230668411 shutdown_connections 2023-12-14T18:26:21.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 --2- 172.21.15.93:0/2230668411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e50105fa0 0x7f6e50108380 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 --2- 172.21.15.93:0/2230668411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e50100d60 0x7f6e50103140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 --2- 172.21.15.93:0/2230668411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50103680 0x7f6e50105a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:21.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 -- 172.21.15.93:0/2230668411 >> 172.21.15.93:0/2230668411 conn(0x7f6e500fa720 msgr2=0x7f6e500fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:21.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 -- 172.21.15.93:0/2230668411 shutdown_connections 2023-12-14T18:26:21.997 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 -- 172.21.15.93:0/2230668411 wait complete. 2023-12-14T18:26:21.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 Processor -- start 2023-12-14T18:26:21.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 -- start start 2023-12-14T18:26:21.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e50100d60 0x7f6e501960d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e50103680 0x7f6e50196610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50105fa0 0x7f6e5019d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:21.999 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5019dc50 con 0x7f6e50100d60 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5019dd90 con 0x7f6e50103680 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e5860e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e5019df20 con 0x7f6e50105fa0 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.995+0000 7f6e563aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e50100d60 0x7f6e501960d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e563aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e50100d60 0x7f6e501960d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42694/0 (socket says 172.21.15.93:42694) 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e563aa700 1 -- 172.21.15.93:0/1772668919 learned_addr learned my addr 172.21.15.93:0/1772668919 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e55ba9700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e50103680 0x7f6e50196610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e56bab700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50105fa0 0x7f6e5019d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e563aa700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50105fa0 msgr2=0x7f6e5019d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e563aa700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6e50105fa0 0x7f6e5019d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e563aa700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e50103680 msgr2=0x7f6e50196610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e563aa700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e50103680 0x7f6e50196610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e563aa700 1 -- 172.21.15.93:0/1772668919 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e4c009520 con 0x7f6e50100d60 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e55ba9700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6e50103680 0x7f6e50196610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:22.000 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e563aa700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e50100d60 0x7f6e501960d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00eb30 tx=0x7f6e4c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:22.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c009880 con 0x7f6e50100d60 2023-12-14T18:26:22.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6e4c0044e0 con 0x7f6e50100d60 2023-12-14T18:26:22.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e48009d00 con 0x7f6e50100d60 2023-12-14T18:26:22.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e4c003d80 con 0x7f6e50100d60 2023-12-14T18:26:22.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.996+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e5010bba0 con 0x7f6e50100d60 2023-12-14T18:26:22.001 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.997+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e5010bd30 con 0x7f6e50100d60 2023-12-14T18:26:22.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:21.999+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 19) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f6e4c003f00 con 0x7f6e50100d60 2023-12-14T18:26:22.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.000+0000 7f6e477fe700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 0x7f6e3c0a3650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:22.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.000+0000 7f6e55ba9700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 msgr2=0x7f6e3c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3802499407 2023-12-14T18:26:22.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.000+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(86..86 src has 1..86) v4 ==== 9020+0+0 (secure 0 0 0) 0x7f6e4c014070 con 0x7f6e50100d60 2023-12-14T18:26:22.002 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.000+0000 7f6e55ba9700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 0x7f6e3c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-14T18:26:22.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.003+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6e4c096760 con 0x7f6e50100d60 2023-12-14T18:26:22.092 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:21.774+0000 7f34af70e200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:26:22.125 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:21.776+0000 7f5a10709200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-14T18:26:22.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.201+0000 7f6e55ba9700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 msgr2=0x7f6e3c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3802499407 2023-12-14T18:26:22.202 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.201+0000 7f6e55ba9700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 0x7f6e3c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-14T18:26:22.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.268+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 --> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e50060f30 con 0x7f6e3c0a11a0 2023-12-14T18:26:22.381 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:22 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/4197811593' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-14T18:26:22.381 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:22 smithi097 ceph-mon[43486]: mgrmap e19: x(active, since 6m), standbys: z, y 2023-12-14T18:26:22.381 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:22 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1358789782' entity='client.admin' 2023-12-14T18:26:22.381 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:22 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2404951887' entity='client.admin' 2023-12-14T18:26:22.394 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:22 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/4197811593' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-14T18:26:22.394 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:22 smithi160 ceph-mon[43840]: mgrmap e19: x(active, since 6m), standbys: z, y 2023-12-14T18:26:22.394 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:22 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1358789782' entity='client.admin' 2023-12-14T18:26:22.394 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:22 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2404951887' entity='client.admin' 2023-12-14T18:26:22.435 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:22 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/4197811593' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2023-12-14T18:26:22.435 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:22 smithi093 ceph-mon[42070]: mgrmap e19: x(active, since 6m), standbys: z, y 2023-12-14T18:26:22.435 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:22 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1358789782' entity='client.admin' 2023-12-14T18:26:22.435 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:22 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2404951887' entity='client.admin' 2023-12-14T18:26:22.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.602+0000 7f6e55ba9700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 msgr2=0x7f6e3c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3802499407 2023-12-14T18:26:22.603 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:22.602+0000 7f6e55ba9700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 0x7f6e3c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-14T18:26:22.699 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:22.379+0000 7f34af70e200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:26:22.699 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:22.612+0000 7f34af70e200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:26:22.717 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:22.393+0000 7f5a10709200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:26:22.717 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:22.630+0000 7f5a10709200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:26:22.755 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:22.433+0000 7fc63c20f200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:26:22.755 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:22.668+0000 7fc63c20f200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-14T18:26:22.755 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:22.753+0000 7fc63c20f200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:26:22.995 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:22.716+0000 7f5a10709200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:26:22.996 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:22.827+0000 7f5a10709200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:26:22.996 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:22.918+0000 7f5a10709200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:26:23.064 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:22.863+0000 7fc63c20f200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:26:23.064 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:22.953+0000 7fc63c20f200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:26:23.092 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:22.697+0000 7f34af70e200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-14T18:26:23.092 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:22.807+0000 7f34af70e200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-14T18:26:23.092 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:22.897+0000 7f34af70e200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-14T18:26:23.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:23.090+0000 7f34af70e200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:26:23.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:23.177+0000 7f34af70e200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:26:23.404 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:23.403+0000 7f6e55ba9700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 msgr2=0x7f6e3c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3802499407 2023-12-14T18:26:23.405 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:23.403+0000 7f6e55ba9700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 0x7f6e3c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-14T18:26:23.496 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:23.117+0000 7f5a10709200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:26:23.496 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:23.203+0000 7f5a10709200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:26:23.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:23 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:23.148+0000 7fc63c20f200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-14T18:26:23.565 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:23 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:23.232+0000 7fc63c20f200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-14T18:26:23.808 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:23.544+0000 7f34af70e200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:26:23.845 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:23.577+0000 7f5a10709200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:26:23.869 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:23 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:23.602+0000 7fc63c20f200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-14T18:26:24.088 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:23.806+0000 7f34af70e200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:26:24.130 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:23.844+0000 7f5a10709200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:26:24.151 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:23 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:23.867+0000 7fc63c20f200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-14T18:26:24.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:24.086+0000 7f34af70e200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:26:24.343 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:24.179+0000 7f34af70e200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:26:24.496 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:24 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:24.129+0000 7f5a10709200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:26:24.496 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:24 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:24.225+0000 7f5a10709200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:26:24.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:24 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:24.149+0000 7fc63c20f200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-12-14T18:26:24.565 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:24 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:24.245+0000 7fc63c20f200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-14T18:26:25.006 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:25.004+0000 7f6e55ba9700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 msgr2=0x7f6e3c0a3650 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.93:6800/3802499407 2023-12-14T18:26:25.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:25.004+0000 7f6e55ba9700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 0x7f6e3c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-14T18:26:25.246 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:24 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:24.984+0000 7f5a10709200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:26:25.246 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:25.079+0000 7f5a10709200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:26:25.246 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:25.228+0000 7f5a10709200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:26:25.315 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:25 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:24.999+0000 7fc63c20f200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:26:25.315 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:25 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:25.096+0000 7fc63c20f200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:26:25.315 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:25 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:25.244+0000 7fc63c20f200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:26:25.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:24.927+0000 7f34af70e200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-14T18:26:25.342 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:25.023+0000 7f34af70e200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-14T18:26:25.343 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:25.169+0000 7f34af70e200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-14T18:26:26.092 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:25.788+0000 7f34af70e200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:26:26.093 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:25.971+0000 7f34af70e200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:26:26.245 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:25.862+0000 7f5a10709200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:26:26.245 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:26.047+0000 7f5a10709200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:26:26.254 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:25 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:25.868+0000 7fc63c20f200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-12-14T18:26:26.254 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:26 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:26.055+0000 7fc63c20f200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-14T18:26:26.496 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:26.243+0000 7f5a10709200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:26:26.564 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:26 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:26.252+0000 7fc63c20f200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:26:26.592 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:26 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:26.165+0000 7f34af70e200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-14T18:26:27.246 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:26.957+0000 7f5a10709200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:26:27.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:26 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:26.964+0000 7fc63c20f200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:26:27.340 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:26 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:26.870+0000 7f34af70e200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-14T18:26:27.592 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:27.338+0000 7f34af70e200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:26:27.592 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:27.428+0000 7f34af70e200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:26:27.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:27.690+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mgrmap(e 20) v1 ==== 135732+0+0 (secure 0 0 0) 0x7f6e4c0041b0 con 0x7f6e50100d60 2023-12-14T18:26:27.695 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.439+0000 7fc63c20f200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:26:27.695 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.529+0000 7fc63c20f200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:26:27.697 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:27.436+0000 7f5a10709200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-14T18:26:27.697 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:27.525+0000 7f5a10709200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-14T18:26:27.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:27.751+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mon.0 v2:172.21.15.93:3300/0 8 ==== mgrmap(e 21) v1 ==== 135400+0+0 (secure 0 0 0) 0x7f6e4c0cab00 con 0x7f6e50100d60 2023-12-14T18:26:27.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:27.752+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 msgr2=0x7f6e3c0a3650 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:26:27.753 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:27.752+0000 7f6e477fe700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3802499407,v1:172.21.15.93:6801/3802499407] conn(0x7f6e3c0a11a0 0x7f6e3c0a3650 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:27.887 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:26:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y[44934]: 2023-12-14T18:26:27.618+0000 7f34af70e200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:26:27.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:27 smithi097 ceph-mon[43486]: Standby manager daemon y restarted 2023-12-14T18:26:27.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:27 smithi097 ceph-mon[43486]: Standby manager daemon y started 2023-12-14T18:26:27.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:27 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:26:27.887 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:27 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:26:27.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:27 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:26:27.888 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:27 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:26:27.947 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.727+0000 7fc63c20f200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:26:27.947 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.944+0000 7fc599e5f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:27.947 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.944+0000 7fc599e5f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:27.947 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.944+0000 7fc599e5f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:27.947 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.944+0000 7fc599e5f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:27.947 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.944+0000 7fc599e5f700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:27.949 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-mon[42070]: Standby manager daemon y restarted 2023-12-14T18:26:27.949 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-mon[42070]: Standby manager daemon y started 2023-12-14T18:26:27.949 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:26:27.949 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:26:27.949 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:26:27.949 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:26:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:27 smithi160 ceph-mon[43840]: Standby manager daemon y restarted 2023-12-14T18:26:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:27 smithi160 ceph-mon[43840]: Standby manager daemon y started 2023-12-14T18:26:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:27 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2023-12-14T18:26:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:27 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:26:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:27 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2023-12-14T18:26:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:27 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.97:0/14299378' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:26:27.997 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:26:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z[45672]: 2023-12-14T18:26:27.720+0000 7f5a10709200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-14T18:26:28.315 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.946+0000 7fc596e59700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:28.315 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.946+0000 7fc596e59700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:28.315 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.946+0000 7fc596e59700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:28.315 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.946+0000 7fc596e59700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:28.315 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:26:27 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:26:27.946+0000 7fc596e59700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:26:28.761 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.759+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mon.0 v2:172.21.15.93:3300/0 9 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f6e4c0cd9b0 con 0x7f6e50100d60 2023-12-14T18:26:28.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.759+0000 7f6e477fe700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6e3c00eb60 0x7f6e3c00ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:28.762 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.759+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 --> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e50060f30 con 0x7f6e3c00eb60 2023-12-14T18:26:28.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.762+0000 7f6e55ba9700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6e3c00eb60 0x7f6e3c00ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:28.767 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.765+0000 7f6e55ba9700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6e3c00eb60 0x7f6e3c00ef30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009c30 tx=0x7f6e40009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: mgrmap e20: x(active, since 6m), standbys: y, z 2023-12-14T18:26:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: Standby manager daemon z restarted 2023-12-14T18:26:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: Standby manager daemon z started 2023-12-14T18:26:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:26:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:26:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:26:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:26:28.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: Active manager daemon x restarted 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: Activating manager daemon x 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: osdmap e87: 12 total, 12 up, 12 in 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: mgrmap e21: x(active, starting, since 0.0119006s), standbys: y, z 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:26:28.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:26:28.817 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: Manager daemon x is now available 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:26:28.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:28 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:26:28.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.822+0000 7f6e477fe700 1 -- 172.21.15.93:0/1772668919 <== mgr.25100 v2:172.21.15.93:6800/3274300600 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6e50060f30 con 0x7f6e3c00eb60 2023-12-14T18:26:28.824 INFO:teuthology.orchestra.run.smithi093.stdout:Schedule set for path / 2023-12-14T18:26:28.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.825+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6e3c00eb60 msgr2=0x7f6e3c00ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:28.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.825+0000 7f6e5860e700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6e3c00eb60 0x7f6e3c00ef30 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6e40009c30 tx=0x7f6e40009260 comp rx=0 tx=0).stop 2023-12-14T18:26:28.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.825+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e50100d60 msgr2=0x7f6e501960d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:28.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.825+0000 7f6e5860e700 1 --2- 172.21.15.93:0/1772668919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6e50100d60 0x7f6e501960d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f6e4c00eb30 tx=0x7f6e4c00ee40 comp rx=0 tx=0).stop 2023-12-14T18:26:28.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.825+0000 7f6e56bab700 1 -- 172.21.15.93:0/1772668919 reap_dead start 2023-12-14T18:26:28.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.825+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 shutdown_connections 2023-12-14T18:26:28.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.825+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 >> 172.21.15.93:0/1772668919 conn(0x7f6e500fa720 msgr2=0x7f6e501019c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:28.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.825+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 shutdown_connections 2023-12-14T18:26:28.827 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.825+0000 7f6e5860e700 1 -- 172.21.15.93:0/1772668919 wait complete. 2023-12-14T18:26:28.840 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6M3h' 2023-12-14T18:26:28.984 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.982+0000 7fb204267700 1 -- 172.21.15.93:0/640003249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0ff260 msgr2=0x7fb1fc0ff6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:28.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.982+0000 7fb204267700 1 --2- 172.21.15.93:0/640003249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0ff260 0x7fb1fc0ff6d0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fb1ec009990 tx=0x7fb1ec009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:28.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.982+0000 7fb204267700 1 -- 172.21.15.93:0/640003249 shutdown_connections 2023-12-14T18:26:28.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.982+0000 7fb204267700 1 --2- 172.21.15.93:0/640003249 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1fc0fe500 0x7fb1fc0fe910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:28.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.982+0000 7fb204267700 1 --2- 172.21.15.93:0/640003249 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1fc0ffc10 0x7fb1fc1044c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:28.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.982+0000 7fb204267700 1 --2- 172.21.15.93:0/640003249 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0ff260 0x7fb1fc0ff6d0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:28.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.982+0000 7fb204267700 1 -- 172.21.15.93:0/640003249 >> 172.21.15.93:0/640003249 conn(0x7fb1fc0f9ea0 msgr2=0x7fb1fc0fc310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:28.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.983+0000 7fb204267700 1 -- 172.21.15.93:0/640003249 shutdown_connections 2023-12-14T18:26:28.985 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.983+0000 7fb204267700 1 -- 172.21.15.93:0/640003249 wait complete. 2023-12-14T18:26:28.987 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.983+0000 7fb204267700 1 Processor -- start 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.983+0000 7fb204267700 1 -- start start 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb204267700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1fc0fe500 0x7fb1fc19c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb204267700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0ff260 0x7fb1fc19cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb204267700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1fc0ffc10 0x7fb1fc1a3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb204267700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc1a4190 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb204267700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc1a42d0 con 0x7fb1fc0fe500 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb204267700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc1a4460 con 0x7fb1fc0ff260 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202804700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1fc0ffc10 0x7fb1fc1a3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202804700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1fc0ffc10 0x7fb1fc1a3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42868/0 (socket says 172.21.15.93:42868) 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202804700 1 -- 172.21.15.93:0/2981751990 learned_addr learned my addr 172.21.15.93:0/2981751990 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb201802700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0ff260 0x7fb1fc19cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:28.988 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202003700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1fc0fe500 0x7fb1fc19c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202804700 1 -- 172.21.15.93:0/2981751990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0ff260 msgr2=0x7fb1fc19cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202804700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0ff260 0x7fb1fc19cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202804700 1 -- 172.21.15.93:0/2981751990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1fc0fe500 msgr2=0x7fb1fc19c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202804700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1fc0fe500 0x7fb1fc19c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202804700 1 -- 172.21.15.93:0/2981751990 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f8009520 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb201802700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0ff260 0x7fb1fc19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.984+0000 7fb202003700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1fc0fe500 0x7fb1fc19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.985+0000 7fb202804700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1fc0ffc10 0x7fb1fc1a3bc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400ef30 tx=0x7fb1f400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.985+0000 7fb1f37fe700 1 -- 172.21.15.93:0/2981751990 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f4018070 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.985+0000 7fb1f37fe700 1 -- 172.21.15.93:0/2981751990 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb1f400f040 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.985+0000 7fb1f37fe700 1 -- 172.21.15.93:0/2981751990 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f400cd00 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.985+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1ec0095f0 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.985+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc112260 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.989 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.986+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc067380 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.987+0000 7fb1f37fe700 1 -- 172.21.15.93:0/2981751990 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7fb1f4003a00 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.988+0000 7fb1f37fe700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb1e80a0f50 0x7fb1e80a3400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:28.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.988+0000 7fb1f37fe700 1 -- 172.21.15.93:0/2981751990 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb1f40cce20 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.989+0000 7fb202003700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb1e80a0f50 0x7fb1e80a3400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:28.991 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.989+0000 7fb202003700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb1e80a0f50 0x7fb1e80a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009fd0 tx=0x7fb1f8009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:28.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:28.992+0000 7fb1f37fe700 1 -- 172.21.15.93:0/2981751990 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fb1f40964e0 con 0x7fb1fc0ffc10 2023-12-14T18:26:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: mgrmap e20: x(active, since 6m), standbys: y, z 2023-12-14T18:26:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: Standby manager daemon z restarted 2023-12-14T18:26:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: Standby manager daemon z started 2023-12-14T18:26:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:26:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:26:28.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: Active manager daemon x restarted 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: Activating manager daemon x 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: osdmap e87: 12 total, 12 up, 12 in 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: mgrmap e21: x(active, starting, since 0.0119006s), standbys: y, z 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:26:28.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:26:28.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: Manager daemon x is now available 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:26:28.999 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:28 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:26:29.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: mgrmap e20: x(active, since 6m), standbys: y, z 2023-12-14T18:26:29.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: Standby manager daemon z restarted 2023-12-14T18:26:29.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: Standby manager daemon z started 2023-12-14T18:26:29.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2023-12-14T18:26:29.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.? 172.21.15.160:0/2490025739' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: Active manager daemon x restarted 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: Activating manager daemon x 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: osdmap e87: 12 total, 12 up, 12 in 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: mgrmap e21: x(active, starting, since 0.0119006s), standbys: y, z 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2023-12-14T18:26:29.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:26:29.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2023-12-14T18:26:29.096 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2023-12-14T18:26:29.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2023-12-14T18:26:29.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2023-12-14T18:26:29.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2023-12-14T18:26:29.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2023-12-14T18:26:29.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: Manager daemon x is now available 2023-12-14T18:26:29.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:29.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:29.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2023-12-14T18:26:29.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:28 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2023-12-14T18:26:29.241 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.239+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 --> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}) v1 -- 0x7fb1fc0fc220 con 0x7fb1e80a0f50 2023-12-14T18:26:29.305 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.304+0000 7fb1f37fe700 1 -- 172.21.15.93:0/2981751990 <== mgr.25100 v2:172.21.15.93:6800/3274300600 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+26 (secure 0 0 0) 0x7fb1fc0fc220 con 0x7fb1e80a0f50 2023-12-14T18:26:29.306 INFO:teuthology.orchestra.run.smithi093.stdout:Retention added to path / 2023-12-14T18:26:29.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb1e80a0f50 msgr2=0x7fb1e80a3400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb1e80a0f50 0x7fb1e80a3400 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb1f8009fd0 tx=0x7fb1f8009260 comp rx=0 tx=0).stop 2023-12-14T18:26:29.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1fc0ffc10 msgr2=0x7fb1fc1a3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1fc0ffc10 0x7fb1fc1a3bc0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fb1f400ef30 tx=0x7fb1f400c3a0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 shutdown_connections 2023-12-14T18:26:29.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb1e80a0f50 0x7fb1e80a3400 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb1fc0ffc10 0x7fb1fc1a3bc0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.308 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb1fc0fe500 0x7fb1fc19c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 --2- 172.21.15.93:0/2981751990 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb1fc0ff260 0x7fb1fc19cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 >> 172.21.15.93:0/2981751990 conn(0x7fb1fc0f9ea0 msgr2=0x7fb1fc0fab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:29.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 shutdown_connections 2023-12-14T18:26:29.309 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.306+0000 7fb204267700 1 -- 172.21.15.93:0/2981751990 wait complete. 2023-12-14T18:26:29.321 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 -- 172.21.15.93:0/2035229890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c100cc0 msgr2=0x7f595c1030a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 --2- 172.21.15.93:0/2035229890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c100cc0 0x7f595c1030a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f5958009990 tx=0x7f5958009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 -- 172.21.15.93:0/2035229890 shutdown_connections 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 --2- 172.21.15.93:0/2035229890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c100cc0 0x7f595c1030a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 --2- 172.21.15.93:0/2035229890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f595c105f00 0x7f595c108360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 --2- 172.21.15.93:0/2035229890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f595c1035e0 0x7f595c1059c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 -- 172.21.15.93:0/2035229890 >> 172.21.15.93:0/2035229890 conn(0x7f595c0fa720 msgr2=0x7f595c0fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 -- 172.21.15.93:0/2035229890 shutdown_connections 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 -- 172.21.15.93:0/2035229890 wait complete. 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 Processor -- start 2023-12-14T18:26:29.454 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.450+0000 7f5963704700 1 -- start start 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5963704700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f595c100cc0 0x7f595c195f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5963704700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c1035e0 0x7f595c196480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5963704700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f595c105f00 0x7f595c19d4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5963704700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c19dac0 con 0x7f595c1035e0 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5963704700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c19dc00 con 0x7f595c100cc0 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5963704700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595c19dd90 con 0x7f595c105f00 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5960c9f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c1035e0 0x7f595c196480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f59614a0700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f595c100cc0 0x7f595c195f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5960c9f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c1035e0 0x7f595c196480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42890/0 (socket says 172.21.15.93:42890) 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5960c9f700 1 -- 172.21.15.93:0/3189388551 learned_addr learned my addr 172.21.15.93:0/3189388551 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5961ca1700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f595c105f00 0x7f595c19d4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5960c9f700 1 -- 172.21.15.93:0/3189388551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f595c105f00 msgr2=0x7f595c19d4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.455 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5960c9f700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f595c105f00 0x7f595c19d4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5960c9f700 1 -- 172.21.15.93:0/3189388551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f595c100cc0 msgr2=0x7f595c195f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5960c9f700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f595c100cc0 0x7f595c195f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5960c9f700 1 -- 172.21.15.93:0/3189388551 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59580095f0 con 0x7f595c1035e0 2023-12-14T18:26:29.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5960c9f700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c1035e0 0x7f595c196480 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f594c00e9a0 tx=0x7f594c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:29.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f59527fc700 1 -- 172.21.15.93:0/3189388551 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c009790 con 0x7f595c1035e0 2023-12-14T18:26:29.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f59527fc700 1 -- 172.21.15.93:0/3189388551 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f594c010430 con 0x7f595c1035e0 2023-12-14T18:26:29.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f59527fc700 1 -- 172.21.15.93:0/3189388551 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c018680 con 0x7f595c1035e0 2023-12-14T18:26:29.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595c10bbc0 con 0x7f595c1035e0 2023-12-14T18:26:29.456 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.451+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f595c10ba30 con 0x7f595c1035e0 2023-12-14T18:26:29.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.452+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595c19e430 con 0x7f595c1035e0 2023-12-14T18:26:29.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.453+0000 7f59527fc700 1 -- 172.21.15.93:0/3189388551 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f594c0105a0 con 0x7f595c1035e0 2023-12-14T18:26:29.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.454+0000 7f59527fc700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59480a0fd0 0x7f59480a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.454+0000 7f59527fc700 1 -- 172.21.15.93:0/3189388551 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f594c014070 con 0x7f595c1035e0 2023-12-14T18:26:29.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.454+0000 7f59614a0700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59480a0fd0 0x7f59480a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.457 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.454+0000 7f59614a0700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59480a0fd0 0x7f59480a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f595800b2e0 tx=0x7f5958005fb0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:29.459 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.458+0000 7f59527fc700 1 -- 172.21.15.93:0/3189388551 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f594c0962c0 con 0x7f595c1035e0 2023-12-14T18:26:29.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.686+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 --> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f595c19e9e0 con 0x7f59480a0fd0 2023-12-14T18:26:29.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.694+0000 7f59527fc700 1 -- 172.21.15.93:0/3189388551 <== mgr.25100 v2:172.21.15.93:6800/3274300600 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+283 (secure 0 0 0) 0x7f595c19e9e0 con 0x7f59480a0fd0 2023-12-14T18:26:29.696 INFO:teuthology.orchestra.run.smithi093.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-12-14T00:00:00", "created": "2023-12-14T18:26:28", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2023-12-14T18:26:29.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59480a0fd0 msgr2=0x7f59480a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59480a0fd0 0x7f59480a3480 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f595800b2e0 tx=0x7f5958005fb0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c1035e0 msgr2=0x7f595c196480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c1035e0 0x7f595c196480 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f594c00e9a0 tx=0x7f594c00ecb0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 shutdown_connections 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f595c105f00 0x7f595c19d4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f595c100cc0 0x7f595c195f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59480a0fd0 0x7f59480a3480 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 --2- 172.21.15.93:0/3189388551 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f595c1035e0 0x7f595c196480 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 >> 172.21.15.93:0/3189388551 conn(0x7f595c0fa720 msgr2=0x7f595c102180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:29.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 shutdown_connections 2023-12-14T18:26:29.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.697+0000 7f5963704700 1 -- 172.21.15.93:0/3189388551 wait complete. 2023-12-14T18:26:29.712 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-14T18:26:29.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.843+0000 7f1cfd399700 1 -- 172.21.15.93:0/1838596024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf8100cc0 msgr2=0x7f1cf8107580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.843+0000 7f1cfd399700 1 --2- 172.21.15.93:0/1838596024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf8100cc0 0x7f1cf8107580 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f1cf000b0a0 tx=0x7f1cf000b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.843+0000 7f1cfd399700 1 -- 172.21.15.93:0/1838596024 shutdown_connections 2023-12-14T18:26:29.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.843+0000 7f1cfd399700 1 --2- 172.21.15.93:0/1838596024 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf8100cc0 0x7f1cf8107580 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.848 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.843+0000 7f1cfd399700 1 --2- 172.21.15.93:0/1838596024 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf80ff140 0x7f1cf80ff550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.843+0000 7f1cfd399700 1 --2- 172.21.15.93:0/1838596024 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cf8100330 0x7f1cf8100780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.843+0000 7f1cfd399700 1 -- 172.21.15.93:0/1838596024 >> 172.21.15.93:0/1838596024 conn(0x7f1cf80fa720 msgr2=0x7f1cf80fcb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.843+0000 7f1cfd399700 1 -- 172.21.15.93:0/1838596024 shutdown_connections 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.844+0000 7f1cfd399700 1 -- 172.21.15.93:0/1838596024 wait complete. 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.844+0000 7f1cfd399700 1 Processor -- start 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.844+0000 7f1cfd399700 1 -- start start 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.844+0000 7f1cfd399700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf80ff140 0x7f1cf81030d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.844+0000 7f1cfd399700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cf8100330 0x7f1cf8101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.844+0000 7f1cfd399700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf8100cc0 0x7f1cf8101c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.844+0000 7f1cfd399700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf81023f0 con 0x7f1cf80ff140 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.844+0000 7f1cfd399700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf8102560 con 0x7f1cf8100330 2023-12-14T18:26:29.849 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.844+0000 7f1cfd399700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cf81a0110 con 0x7f1cf8100cc0 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cf6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf80ff140 0x7f1cf81030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cee7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cf8100330 0x7f1cf8101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cf6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf80ff140 0x7f1cf81030d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:42906/0 (socket says 172.21.15.93:42906) 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cf6ffd700 1 -- 172.21.15.93:0/3365955372 learned_addr learned my addr 172.21.15.93:0/3365955372 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cf77fe700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf8100cc0 0x7f1cf8101c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cee7fc700 1 -- 172.21.15.93:0/3365955372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf8100cc0 msgr2=0x7f1cf8101c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cee7fc700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf8100cc0 0x7f1cf8101c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cee7fc700 1 -- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf80ff140 msgr2=0x7f1cf81030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cee7fc700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf80ff140 0x7f1cf81030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cee7fc700 1 -- 172.21.15.93:0/3365955372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cf0009d00 con 0x7f1cf8100330 2023-12-14T18:26:29.850 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cf6ffd700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf80ff140 0x7f1cf81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:29.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cee7fc700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cf8100330 0x7f1cf8101710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000e9f0 tx=0x7f1ce000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:29.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cf4ff9700 1 -- 172.21.15.93:0/3365955372 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce000eee0 con 0x7f1cf8100330 2023-12-14T18:26:29.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cf4ff9700 1 -- 172.21.15.93:0/3365955372 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ce000cbd0 con 0x7f1cf8100330 2023-12-14T18:26:29.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cf810aab0 con 0x7f1cf8100330 2023-12-14T18:26:29.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cf4ff9700 1 -- 172.21.15.93:0/3365955372 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ce0018680 con 0x7f1cf8100330 2023-12-14T18:26:29.851 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.845+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cf810a920 con 0x7f1cf8100330 2023-12-14T18:26:29.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.847+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cf81a0850 con 0x7f1cf8100330 2023-12-14T18:26:29.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.848+0000 7f1cf4ff9700 1 -- 172.21.15.93:0/3365955372 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 22) v1 ==== 135527+0+0 (secure 0 0 0) 0x7f1ce000cd40 con 0x7f1cf8100330 2023-12-14T18:26:29.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.848+0000 7f1cf4ff9700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1ce40a0fd0 0x7f1ce40a3480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:29.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.849+0000 7f1cf6ffd700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1ce40a0fd0 0x7f1ce40a3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:29.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.849+0000 7f1cf4ff9700 1 -- 172.21.15.93:0/3365955372 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f1ce0014070 con 0x7f1cf8100330 2023-12-14T18:26:29.852 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.849+0000 7f1cf6ffd700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1ce40a0fd0 0x7f1ce40a3480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8005950 tx=0x7f1ce8009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:29.855 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:29.853+0000 7f1cf4ff9700 1 -- 172.21.15.93:0/3365955372 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1ce0051730 con 0x7f1cf8100330 2023-12-14T18:26:30.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: mgrmap e22: x(active, since 1.01751s), standbys: y, z 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: [14/Dec/2023:18:26:28] ENGINE Bus STARTING 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: [14/Dec/2023:18:26:28] ENGINE Serving on https://172.21.15.93:7150 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: [14/Dec/2023:18:26:29] ENGINE Serving on http://172.21.15.93:8765 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: [14/Dec/2023:18:26:29] ENGINE Bus STARTED 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:29 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.081 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.079+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 --> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f1cf80687f0 con 0x7f1ce40a0fd0 2023-12-14T18:26:30.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.085+0000 7f1cf4ff9700 1 -- 172.21.15.93:0/3365955372 <== mgr.25100 v2:172.21.15.93:6800/3274300600 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f1cf80687f0 con 0x7f1ce40a0fd0 2023-12-14T18:26:30.087 INFO:teuthology.orchestra.run.smithi093.stdout:/ 1M 6M3h 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.087+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1ce40a0fd0 msgr2=0x7f1ce40a3480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.087+0000 7f1cfd399700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1ce40a0fd0 0x7f1ce40a3480 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f1ce8005950 tx=0x7f1ce8009260 comp rx=0 tx=0).stop 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.087+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cf8100330 msgr2=0x7f1cf8101710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.087+0000 7f1cfd399700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cf8100330 0x7f1cf8101710 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f1ce000e9f0 tx=0x7f1ce000ed00 comp rx=0 tx=0).stop 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.088+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 shutdown_connections 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.088+0000 7f1cfd399700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1ce40a0fd0 0x7f1ce40a3480 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.088+0000 7f1cfd399700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1cf80ff140 0x7f1cf81030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.088+0000 7f1cfd399700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1cf8100cc0 0x7f1cf8101c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.088+0000 7f1cfd399700 1 --2- 172.21.15.93:0/3365955372 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1cf8100330 0x7f1cf8101710 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.088+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 >> 172.21.15.93:0/3365955372 conn(0x7f1cf80fa720 msgr2=0x7f1cf80680e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:30.090 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.088+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 shutdown_connections 2023-12-14T18:26:30.091 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:26:30.088+0000 7f1cfd399700 1 -- 172.21.15.93:0/3365955372 wait complete. 2023-12-14T18:26:30.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: mgrmap e22: x(active, since 1.01751s), standbys: y, z 2023-12-14T18:26:30.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: [14/Dec/2023:18:26:28] ENGINE Bus STARTING 2023-12-14T18:26:30.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: [14/Dec/2023:18:26:28] ENGINE Serving on https://172.21.15.93:7150 2023-12-14T18:26:30.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: [14/Dec/2023:18:26:29] ENGINE Serving on http://172.21.15.93:8765 2023-12-14T18:26:30.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: [14/Dec/2023:18:26:29] ENGINE Bus STARTED 2023-12-14T18:26:30.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:29 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.102 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2023-12-14T18:26:30.131 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2023-12-14T18:26:30.142 INFO:teuthology.run_tasks:Running task background_exec... 2023-12-14T18:26:30.183 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:26:30.183 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2023-12-14T18:26:30.185 INFO:teuthology.run_tasks:Running task workunit... 2023-12-14T18:26:30.222 INFO:tasks.workunit:Pulling workunits from ref bf026da9d69000f3d4a9d16360156095d6612a2d 2023-12-14T18:26:30.223 INFO:tasks.workunit:Making a separate scratch dir for every client... 2023-12-14T18:26:30.223 INFO:tasks.workunit:timeout=3h 2023-12-14T18:26:30.223 INFO:tasks.workunit:cleanup=True 2023-12-14T18:26:30.223 DEBUG:teuthology.orchestra.run.smithi093:> stat -- /home/ubuntu/cephtest/mnt.0 2023-12-14T18:26:30.241 INFO:teuthology.orchestra.run.smithi093.stdout: File: /home/ubuntu/cephtest/mnt.0 2023-12-14T18:26:30.241 INFO:teuthology.orchestra.run.smithi093.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2023-12-14T18:26:30.241 INFO:teuthology.orchestra.run.smithi093.stdout:Device: 7ch/124d Inode: 1099511628283 Links: 2 2023-12-14T18:26:30.241 INFO:teuthology.orchestra.run.smithi093.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-14T18:26:30.242 INFO:teuthology.orchestra.run.smithi093.stdout:Context: system_u:object_r:cephfs_t:s0 2023-12-14T18:26:30.242 INFO:teuthology.orchestra.run.smithi093.stdout:Access: 2023-12-14 18:25:02.923010297 +0000 2023-12-14T18:26:30.242 INFO:teuthology.orchestra.run.smithi093.stdout:Modify: 2023-12-14 18:25:02.923010297 +0000 2023-12-14T18:26:30.242 INFO:teuthology.orchestra.run.smithi093.stdout:Change: 2023-12-14 18:26:19.531708978 +0000 2023-12-14T18:26:30.242 INFO:teuthology.orchestra.run.smithi093.stdout: Birth: 2023-12-14 18:25:02.923010297 +0000 2023-12-14T18:26:30.242 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2023-12-14T18:26:30.242 DEBUG:teuthology.orchestra.run.smithi093:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: mgrmap e22: x(active, since 1.01751s), standbys: y, z 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: [14/Dec/2023:18:26:28] ENGINE Bus STARTING 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: [14/Dec/2023:18:26:28] ENGINE Serving on https://172.21.15.93:7150 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: [14/Dec/2023:18:26:29] ENGINE Serving on http://172.21.15.93:8765 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: [14/Dec/2023:18:26:29] ENGINE Bus STARTED 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:29 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:30.316 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/chrisphoffman/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bf026da9d69000f3d4a9d16360156095d6612a2d 2023-12-14T18:26:30.373 INFO:tasks.workunit.client.0.smithi093.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2023-12-14T18:26:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:26:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi097 to 182.5M 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: Unable to set osd_memory_target on smithi097 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:26:31.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:26:31.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:31 smithi160 ceph-mon[43840]: mgrmap e23: x(active, since 3s), standbys: y, z 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi097 to 182.5M 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: Unable to set osd_memory_target on smithi097 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2023-12-14T18:26:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:26:31.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:31 smithi093 ceph-mon[42070]: mgrmap e23: x(active, since 3s), standbys: y, z 2023-12-14T18:26:31.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='client.15528 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6M3h", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:31.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='client.15534 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:31.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: pgmap v4: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:26:31.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='client.25181 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:26:31.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi097 to 182.5M 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: Unable to set osd_memory_target on smithi097 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:26:31.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:26:31.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:26:31.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:31 smithi097 ceph-mon[43486]: mgrmap e23: x(active, since 3s), standbys: y, z 2023-12-14T18:26:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi093 to 182.4M 2023-12-14T18:26:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Unable to set osd_memory_target on smithi093 to 191315353: error parsing value: Value '191315353' is below minimum 939524096 2023-12-14T18:26:32.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi160 to 182.5M 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Unable to set osd_memory_target on smithi160 to 191374131: error parsing value: Value '191374131' is below minimum 939524096 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi093:/etc/ceph/ceph.conf 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi097:/etc/ceph/ceph.conf 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi160:/etc/ceph/ceph.conf 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:26:32.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:32.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:26:32.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:26:32.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:32 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi093 to 182.4M 2023-12-14T18:26:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Unable to set osd_memory_target on smithi093 to 191315353: error parsing value: Value '191315353' is below minimum 939524096 2023-12-14T18:26:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi160 to 182.5M 2023-12-14T18:26:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Unable to set osd_memory_target on smithi160 to 191374131: error parsing value: Value '191374131' is below minimum 939524096 2023-12-14T18:26:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:26:32.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi093:/etc/ceph/ceph.conf 2023-12-14T18:26:32.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi097:/etc/ceph/ceph.conf 2023-12-14T18:26:32.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi160:/etc/ceph/ceph.conf 2023-12-14T18:26:32.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:26:32.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:26:32.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:26:32.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:26:32.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:26:32.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:26:32.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:32.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:26:32.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:26:32.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:32 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi093 to 182.4M 2023-12-14T18:26:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Unable to set osd_memory_target on smithi093 to 191315353: error parsing value: Value '191315353' is below minimum 939524096 2023-12-14T18:26:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi160 to 182.5M 2023-12-14T18:26:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Unable to set osd_memory_target on smithi160 to 191374131: error parsing value: Value '191374131' is below minimum 939524096 2023-12-14T18:26:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:26:32.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi093:/etc/ceph/ceph.conf 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi097:/etc/ceph/ceph.conf 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi160:/etc/ceph/ceph.conf 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.conf 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi097:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi160:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi093:/etc/ceph/ceph.client.admin.keyring 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:26:32.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: Updating smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/config/ceph.client.admin.keyring 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:26:32.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:26:32.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:32 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:26:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:33 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:26:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:33 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:26:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:33 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:26:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:33 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:26:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:33 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:26:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:33 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:26:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:33 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:26:33.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:33 smithi160 ceph-mon[43840]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:26:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:33 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:26:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:33 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:26:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:33 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:26:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:33 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:26:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:33 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:26:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:33 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:26:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:33 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:26:33.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:33 smithi093 ceph-mon[42070]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:26:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:33 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:26:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:33 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:26:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:33 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:26:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:33 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:26:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:33 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:26:33.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:33 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:26:33.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:33 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:26:33.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:33 smithi097 ceph-mon[43486]: pgmap v5: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:26:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:35 smithi160 ceph-mon[43840]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:26:35.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:35 smithi093 ceph-mon[42070]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:26:35.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:35 smithi097 ceph-mon[43486]: pgmap v6: 129 pgs: 129 active+clean; 1.0 MiB data, 80 MiB used, 1.0 TiB / 1.0 TiB avail 2023-12-14T18:26:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:37 smithi160 ceph-mon[43840]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-12-14T18:26:37.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:37 smithi093 ceph-mon[42070]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-12-14T18:26:37.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:37 smithi097 ceph-mon[43486]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 132 KiB/s wr, 51 op/s 2023-12-14T18:26:39.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:39 smithi160 ceph-mon[43840]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 40 op/s 2023-12-14T18:26:39.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:39 smithi093 ceph-mon[42070]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 40 op/s 2023-12-14T18:26:39.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:39 smithi097 ceph-mon[43486]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 103 KiB/s wr, 40 op/s 2023-12-14T18:26:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:41 smithi160 ceph-mon[43840]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-12-14T18:26:41.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:41 smithi093 ceph-mon[42070]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-12-14T18:26:41.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:41 smithi097 ceph-mon[43486]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 84 KiB/s wr, 35 op/s 2023-12-14T18:26:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 -- 172.21.15.93:0/1438651097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d4101c10 msgr2=0x7ff7d410a2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 --2- 172.21.15.93:0/1438651097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d4101c10 0x7ff7d410a2e0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7ff7c8009990 tx=0x7ff7c8009ca0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 -- 172.21.15.93:0/1438651097 shutdown_connections 2023-12-14T18:26:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 --2- 172.21.15.93:0/1438651097 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7d410a820 0x7ff7d410cc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.348 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 --2- 172.21.15.93:0/1438651097 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7d4101300 0x7ff7d41016d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 --2- 172.21.15.93:0/1438651097 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d4101c10 0x7ff7d410a2e0 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 -- 172.21.15.93:0/1438651097 >> 172.21.15.93:0/1438651097 conn(0x7ff7d40763a0 msgr2=0x7ff7d40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 -- 172.21.15.93:0/1438651097 shutdown_connections 2023-12-14T18:26:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 -- 172.21.15.93:0/1438651097 wait complete. 2023-12-14T18:26:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.344+0000 7ff7da769700 1 Processor -- start 2023-12-14T18:26:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7da769700 1 -- start start 2023-12-14T18:26:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7da769700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7d4101300 0x7ff7d41962c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7da769700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7d4101c10 0x7ff7d4196800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.349 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7da769700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d410a820 0x7ff7d419ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7da769700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d419b2a0 con 0x7ff7d410a820 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7da769700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d419b410 con 0x7ff7d4101300 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7da769700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7d419b5d0 con 0x7ff7d4101c10 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d8d06700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d410a820 0x7ff7d419ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7d4101c10 0x7ff7d4196800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d8d06700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d410a820 0x7ff7d419ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50890/0 (socket says 172.21.15.93:50890) 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d8d06700 1 -- 172.21.15.93:0/2879438778 learned_addr learned my addr 172.21.15.93:0/2879438778 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7d4101300 0x7ff7d41962c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d8d06700 1 -- 172.21.15.93:0/2879438778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7d4101c10 msgr2=0x7ff7d4196800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:43.350 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d8d06700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7d4101c10 0x7ff7d4196800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d8d06700 1 -- 172.21.15.93:0/2879438778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7d4101300 msgr2=0x7ff7d41962c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d8d06700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7d4101300 0x7ff7d41962c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d8d06700 1 -- 172.21.15.93:0/2879438778 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7c80095f0 con 0x7ff7d410a820 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d37fe700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7d4101c10 0x7ff7d4196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.345+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7d4101300 0x7ff7d41962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.346+0000 7ff7d8d06700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d410a820 0x7ff7d419ac60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0094f0 tx=0x7ff7cc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.346+0000 7ff7d17fa700 1 -- 172.21.15.93:0/2879438778 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc00ddd0 con 0x7ff7d410a820 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.346+0000 7ff7d17fa700 1 -- 172.21.15.93:0/2879438778 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7cc00f040 con 0x7ff7d410a820 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.346+0000 7ff7da769700 1 -- 172.21.15.93:0/2879438778 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7d41102b0 con 0x7ff7d410a820 2023-12-14T18:26:43.351 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.346+0000 7ff7d17fa700 1 -- 172.21.15.93:0/2879438778 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc015400 con 0x7ff7d410a820 2023-12-14T18:26:43.352 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.346+0000 7ff7da769700 1 -- 172.21.15.93:0/2879438778 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7d4110120 con 0x7ff7d410a820 2023-12-14T18:26:43.359 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.348+0000 7ff7d17fa700 1 -- 172.21.15.93:0/2879438778 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135891+0+0 (secure 0 0 0) 0x7ff7cc004c40 con 0x7ff7d410a820 2023-12-14T18:26:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.349+0000 7ff7d17fa700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff7bc0a1350 0x7ff7bc0a3800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.349+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff7bc0a1350 0x7ff7bc0a3800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.349+0000 7ff7d17fa700 1 -- 172.21.15.93:0/2879438778 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7ff7cc0ccaa0 con 0x7ff7d410a820 2023-12-14T18:26:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.349+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff7bc0a1350 0x7ff7bc0a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005fd0 tx=0x7ff7c4005e30 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.349+0000 7ff7e68f7b80 1 Processor -- start 2023-12-14T18:26:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7e68f7b80 1 -- start start 2023-12-14T18:26:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5ba7e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 0x5649e5ba87e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.360 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 0x5649e5bb54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5a92580 con 0x5649e5a05430 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5b01810 con 0x5649e5bb30f0 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5afaca0 con 0x5649e5ba8370 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5ba7e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5ba7e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50892/0 (socket says 172.21.15.93:50892) 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d3fff700 1 -- 172.21.15.93:0/3639640109 learned_addr learned my addr 172.21.15.93:0/3639640109 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d8d06700 1 --2- 172.21.15.93:0/3639640109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 0x5649e5bb54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d37fe700 1 --2- 172.21.15.93:0/3639640109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 0x5649e5ba87e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d3fff700 1 -- 172.21.15.93:0/3639640109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 msgr2=0x5649e5ba87e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:43.361 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/3639640109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 0x5649e5ba87e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d3fff700 1 -- 172.21.15.93:0/3639640109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 msgr2=0x5649e5bb54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/3639640109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 0x5649e5bb54d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d3fff700 1 -- 172.21.15.93:0/3639640109 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649e597c350 con 0x5649e5a05430 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.350+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/3639640109 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5ba7e30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400a5c0 tx=0x7ff7c4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b83added60bca86 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7ba7fc700 1 -- 172.21.15.93:0/3639640109 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4010da0 con 0x5649e5a05430 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7ba7fc700 1 -- 172.21.15.93:0/3639640109 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c400eec0 con 0x5649e5a05430 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7ba7fc700 1 -- 172.21.15.93:0/3639640109 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400f780 con 0x5649e5a05430 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/3639640109 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 msgr2=0x5649e5ba7e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/3639640109 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5ba7e30 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400a5c0 tx=0x7ff7c4011000 comp rx=0 tx=0).stop 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/3639640109 shutdown_connections 2023-12-14T18:26:43.362 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/3639640109 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5ba7e30 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/3639640109 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 0x5649e5bb54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/3639640109 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 0x5649e5ba87e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/3639640109 >> 172.21.15.93:0/3639640109 conn(0x5649e5b9e890 msgr2=0x5649e5ba4200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/3639640109 shutdown_connections 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.351+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/3639640109 wait complete. 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.352+0000 7ff7e68f7b80 1 -- start start 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.352+0000 7ff7e68f7b80 1 Processor -- start 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.352+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5a05430 0x5649e5bb2a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.352+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 0x5649e5bafd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.352+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5bb30f0 0x5649e5bb02d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.352+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5a96f30 con 0x5649e5bb30f0 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.352+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5a96bd0 con 0x5649e5a05430 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.352+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e597e640 con 0x5649e5ba8370 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7e68f7b80 10 client.0 _ll_register_callbacks cb 0x5649e5aba8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d8d06700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5bb30f0 0x5649e5bb02d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d8d06700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5bb30f0 0x5649e5bb02d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50902/0 (socket says 172.21.15.93:50902) 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d8d06700 1 -- 172.21.15.93:0/311869829 learned_addr learned my addr 172.21.15.93:0/311869829 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5a05430 0x5649e5bb2a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d37fe700 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 0x5649e5bafd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d8d06700 1 -- 172.21.15.93:0/311869829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 msgr2=0x5649e5bafd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d8d06700 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 0x5649e5bafd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d8d06700 1 -- 172.21.15.93:0/311869829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5a05430 msgr2=0x5649e5bb2a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d8d06700 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5a05430 0x5649e5bb2a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d8d06700 1 -- 172.21.15.93:0/311869829 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7c400eba0 con 0x5649e5bb30f0 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d37fe700 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 0x5649e5bafd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7d8d06700 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5bb30f0 0x5649e5bb02d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0c9ed0 tx=0x7ff7cc0b9330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2391031f7c84ad63 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7b37fe700 1 -- 172.21.15.93:0/311869829 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7cc09d740 con 0x5649e5bb30f0 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7b37fe700 1 -- 172.21.15.93:0/311869829 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7ff7cc0a1b60 con 0x5649e5bb30f0 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/311869829 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649e5af3c80 con 0x5649e5bb30f0 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7b37fe700 1 -- 172.21.15.93:0/311869829 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7cc098c00 con 0x5649e5bb30f0 2023-12-14T18:26:43.365 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.353+0000 7ff7e68f7b80 10 client.15552 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/311869829 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5649e5ac1110 con 0x5649e5bb30f0 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7b37fe700 1 -- 172.21.15.93:0/311869829 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7cc0c7070 con 0x5649e5bb30f0 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7b37fe700 1 -- 172.21.15.93:0/311869829 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7cc09ab70 con 0x5649e5bb30f0 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7e68f7b80 10 client.15552 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7e68f7b80 10 client.15552 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7e68f7b80 20 client.15552 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5649e5bc2b20 0x5649e5bc4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7e68f7b80 4 client.15552 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/311869829 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5649e5af39a0 con 0x5649e5bc2b20 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.354+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5649e5bc2b20 0x5649e5bc4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.355+0000 7ff7b37fe700 10 client.15552 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.355+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5649e5bc2b20 0x5649e5bc4f00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9c4edbf4895c1e0d server_cookie=d398ae6ccb679c0b in_seq=0 out_seq=0 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.355+0000 7ff7b37fe700 1 -- 172.21.15.93:0/311869829 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x5649e5af39a0 con 0x5649e5bc2b20 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.355+0000 7ff7b37fe700 10 client.15552 handle_command_reply: tid=0 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.355+0000 7ff7e68f7b80 1 client.15552 shutdown 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.355+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/311869829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5bb30f0 msgr2=0x5649e5bb02d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.355+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5bb30f0 0x5649e5bb02d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0c9ed0 tx=0x7ff7cc0b9330 comp rx=0 tx=0).stop 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.355+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/311869829 shutdown_connections 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.356+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5649e5bc2b20 0x5649e5bc4f00 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.356+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5bb30f0 0x5649e5bb02d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.356+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5a05430 0x5649e5bb2a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.356+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/311869829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5ba8370 0x5649e5bafd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.356+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/311869829 >> 172.21.15.93:0/311869829 conn(0x5649e5b9e890 msgr2=0x5649e5ba3d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.356+0000 7ff7b37fe700 0 client.15552 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.356+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/311869829 shutdown_connections 2023-12-14T18:26:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.356+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/311869829 wait complete. 2023-12-14T18:26:43.368 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.356+0000 7ff7e68f7b80 20 client.15552 trim_cache size 0 max 16384 2023-12-14T18:26:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.371+0000 7ff7e68f7b80 1 Processor -- start 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.371+0000 7ff7e68f7b80 1 -- start start 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.371+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5bb9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.371+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5ba8370 0x5649e5bc4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.371+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5bb30f0 0x5649e5bb2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.371+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5b9e4d0 con 0x5649e5a05430 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.371+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5ba8180 con 0x5649e5ba8370 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.371+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5bb54d0 con 0x5649e5bb30f0 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.371+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5bb9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5bb9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50908/0 (socket says 172.21.15.93:50908) 2023-12-14T18:26:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d3fff700 1 -- 172.21.15.93:0/2656495381 learned_addr learned my addr 172.21.15.93:0/2656495381 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d37fe700 1 --2- 172.21.15.93:0/2656495381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5ba8370 0x5649e5bc4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d8d06700 1 --2- 172.21.15.93:0/2656495381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5bb30f0 0x5649e5bb2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d3fff700 1 -- 172.21.15.93:0/2656495381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5bb30f0 msgr2=0x5649e5bb2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/2656495381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5bb30f0 0x5649e5bb2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d3fff700 1 -- 172.21.15.93:0/2656495381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5ba8370 msgr2=0x5649e5bc4b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/2656495381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5ba8370 0x5649e5bc4b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d3fff700 1 -- 172.21.15.93:0/2656495381 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5649e5ba8bb0 con 0x5649e5a05430 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/2656495381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5bb9cc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005ea0 tx=0x7ff7c4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1aeea523e77d85af server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7baffd700 1 -- 172.21.15.93:0/2656495381 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c402c040 con 0x5649e5a05430 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7baffd700 1 -- 172.21.15.93:0/2656495381 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff7c402d040 con 0x5649e5a05430 2023-12-14T18:26:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.372+0000 7ff7baffd700 1 -- 172.21.15.93:0/2656495381 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c40332b0 con 0x5649e5a05430 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/2656495381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 msgr2=0x5649e5bb9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/2656495381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5bb9cc0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005ea0 tx=0x7ff7c4014000 comp rx=0 tx=0).stop 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/2656495381 shutdown_connections 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/2656495381 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5a05430 0x5649e5bb9cc0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/2656495381 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5bb30f0 0x5649e5bb2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/2656495381 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5ba8370 0x5649e5bc4b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/2656495381 >> 172.21.15.93:0/2656495381 conn(0x5649e5b9e890 msgr2=0x5649e5bbee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/2656495381 shutdown_connections 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/2656495381 wait complete. 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.373+0000 7ff7e68f7b80 1 -- start start 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7e68f7b80 1 Processor -- start 2023-12-14T18:26:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5a05430 0x5649e5c0a940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5ba8370 0x5649e5c0ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7e68f7b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 0x5649e5c0b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5c15af0 con 0x5649e5ba8370 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5c15c60 con 0x5649e5bb30f0 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7e68f7b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5649e5c15e20 con 0x5649e5a05430 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7e68f7b80 10 client.0 _ll_register_callbacks cb 0x7ff7cc0994e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5ba8370 0x5649e5c0ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5ba8370 0x5649e5c0ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50910/0 (socket says 172.21.15.93:50910) 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d37fe700 1 -- 172.21.15.93:0/4253918787 learned_addr learned my addr 172.21.15.93:0/4253918787 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:43.381 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5a05430 0x5649e5c0a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d8d06700 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 0x5649e5c0b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d37fe700 1 -- 172.21.15.93:0/4253918787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5a05430 msgr2=0x5649e5c0a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d37fe700 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5a05430 0x5649e5c0a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d37fe700 1 -- 172.21.15.93:0/4253918787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 msgr2=0x5649e5c0b3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d37fe700 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 0x5649e5c0b3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d37fe700 1 -- 172.21.15.93:0/4253918787 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5649e597c350 con 0x5649e5ba8370 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7d37fe700 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5ba8370 0x5649e5c0ae80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80098e0 tx=0x7ff7c800fc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef1d085079a9602d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.374+0000 7ff7b37fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7b37fe700 1 -- 172.21.15.93:0/4253918787 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7c801d040 con 0x5649e5ba8370 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7b37fe700 1 -- 172.21.15.93:0/4253918787 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7ff7c8003890 con 0x5649e5ba8370 2023-12-14T18:26:43.382 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/4253918787 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5649e5b9bea0 con 0x5649e5ba8370 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7b37fe700 1 -- 172.21.15.93:0/4253918787 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7c8009ee0 con 0x5649e5ba8370 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7e68f7b80 10 client.15558 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/4253918787 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x5649e5928b00 con 0x5649e5ba8370 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7b37fe700 1 -- 172.21.15.93:0/4253918787 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x5649e5b9bea0 con 0x5649e5ba8370 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7b37fe700 1 -- 172.21.15.93:0/4253918787 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7ff7c8025400 con 0x5649e5ba8370 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7e68f7b80 10 client.15558 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7e68f7b80 10 client.15558 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7e68f7b80 20 client.15558 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5649e5c17480 0x5649e5c19890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7e68f7b80 4 client.15558 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.375+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/4253918787 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5649e5949c90 con 0x5649e5c17480 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.376+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5649e5c17480 0x5649e5c19890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:43.383 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.376+0000 7ff7b37fe700 10 client.15558 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:43.384 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.376+0000 7ff7d3fff700 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5649e5c17480 0x5649e5c19890 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a5d2bed74695e66e server_cookie=45009b7bc1be7193 in_seq=0 out_seq=0 2023-12-14T18:26:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.388+0000 7ff7b37fe700 1 -- 172.21.15.93:0/4253918787 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5649e5949c90 con 0x5649e5c17480 2023-12-14T18:26:43.390 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.388+0000 7ff7b37fe700 10 client.15558 handle_command_reply: tid=0 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.388+0000 7ff7e68f7b80 1 client.15558 shutdown 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/4253918787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5ba8370 msgr2=0x5649e5c0ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5ba8370 0x5649e5c0ae80 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7ff7c80098e0 tx=0x7ff7c800fc20 comp rx=0 tx=0).stop 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/4253918787 shutdown_connections 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x5649e5c17480 0x5649e5c19890 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5649e5bb30f0 0x5649e5c0b3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x5649e5a05430 0x5649e5c0a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7b37fe700 0 client.15558 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 --2- 172.21.15.93:0/4253918787 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5649e5ba8370 0x5649e5c0ae80 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/4253918787 >> 172.21.15.93:0/4253918787 conn(0x5649e5b9e890 msgr2=0x5649e5bc4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/4253918787 shutdown_connections 2023-12-14T18:26:43.391 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 1 -- 172.21.15.93:0/4253918787 wait complete. 2023-12-14T18:26:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7e68f7b80 20 client.15558 trim_cache size 0 max 16384 2023-12-14T18:26:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:26:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:26:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:26:43.392 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:26:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7da769700 1 -- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff7bc0a1350 msgr2=0x7ff7bc0a3800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7da769700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff7bc0a1350 0x7ff7bc0a3800 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7ff7c4005fd0 tx=0x7ff7c4005e30 comp rx=0 tx=0).stop 2023-12-14T18:26:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7da769700 1 -- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d410a820 msgr2=0x7ff7d419ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:43.393 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.389+0000 7ff7da769700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d410a820 0x7ff7d419ac60 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7ff7cc0094f0 tx=0x7ff7cc00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.390+0000 7ff7da769700 1 -- 172.21.15.93:0/2879438778 shutdown_connections 2023-12-14T18:26:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.390+0000 7ff7da769700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff7bc0a1350 0x7ff7bc0a3800 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.390+0000 7ff7da769700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff7d410a820 0x7ff7d419ac60 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.390+0000 7ff7da769700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff7d4101300 0x7ff7d41962c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.390+0000 7ff7da769700 1 --2- 172.21.15.93:0/2879438778 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff7d4101c10 0x7ff7d4196800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.390+0000 7ff7da769700 1 -- 172.21.15.93:0/2879438778 >> 172.21.15.93:0/2879438778 conn(0x7ff7d40763a0 msgr2=0x7ff7d40fd8b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.390+0000 7ff7da769700 1 -- 172.21.15.93:0/2879438778 shutdown_connections 2023-12-14T18:26:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:43.390+0000 7ff7da769700 1 -- 172.21.15.93:0/2879438778 wait complete. 2023-12-14T18:26:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:43 smithi160 ceph-mon[43840]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-12-14T18:26:43.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:43 smithi093 ceph-mon[42070]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-12-14T18:26:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:43 smithi097 ceph-mon[43486]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-12-14T18:26:45.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:45 smithi160 ceph-mon[43840]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-12-14T18:26:45.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:45 smithi093 ceph-mon[42070]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-12-14T18:26:45.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:45 smithi097 ceph-mon[43486]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 77 KiB/s wr, 32 op/s 2023-12-14T18:26:47.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:47 smithi160 ceph-mon[43840]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 39 op/s 2023-12-14T18:26:47.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:47 smithi093 ceph-mon[42070]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 39 op/s 2023-12-14T18:26:47.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:47 smithi097 ceph-mon[43486]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 78 KiB/s wr, 39 op/s 2023-12-14T18:26:49.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:49 smithi160 ceph-mon[43840]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-12-14T18:26:49.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:49 smithi093 ceph-mon[42070]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-12-14T18:26:49.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:49 smithi097 ceph-mon[43486]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-12-14T18:26:51.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:51 smithi160 ceph-mon[43840]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-12-14T18:26:51.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:51 smithi093 ceph-mon[42070]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-12-14T18:26:51.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:51 smithi097 ceph-mon[43486]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-12-14T18:26:52.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:52 smithi160 ceph-mon[43840]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-12-14T18:26:52.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:52 smithi093 ceph-mon[42070]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-12-14T18:26:52.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:52 smithi097 ceph-mon[43486]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2023-12-14T18:26:55.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:54 smithi097 ceph-mon[43486]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-14T18:26:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:54 smithi160 ceph-mon[43840]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-14T18:26:55.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:54 smithi093 ceph-mon[42070]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 1.5 KiB/s wr, 10 op/s 2023-12-14T18:26:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.540+0000 7f2961095700 1 -- 172.21.15.93:0/2588790723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c10a810 msgr2=0x7f295c10cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.540+0000 7f2961095700 1 --2- 172.21.15.93:0/2588790723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c10a810 0x7f295c10cc30 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f295400b0a0 tx=0x7f295400b3b0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.540+0000 7f2961095700 1 -- 172.21.15.93:0/2588790723 shutdown_connections 2023-12-14T18:26:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.540+0000 7f2961095700 1 --2- 172.21.15.93:0/2588790723 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c10a810 0x7f295c10cc30 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.540+0000 7f2961095700 1 --2- 172.21.15.93:0/2588790723 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c101440 0x7f295c101810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.540+0000 7f2961095700 1 --2- 172.21.15.93:0/2588790723 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f295c101d50 0x7f295c10a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.540+0000 7f2961095700 1 -- 172.21.15.93:0/2588790723 >> 172.21.15.93:0/2588790723 conn(0x7f295c0763a0 msgr2=0x7f295c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.540+0000 7f2961095700 1 -- 172.21.15.93:0/2588790723 shutdown_connections 2023-12-14T18:26:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.540+0000 7f2961095700 1 -- 172.21.15.93:0/2588790723 wait complete. 2023-12-14T18:26:56.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.541+0000 7f2961095700 1 Processor -- start 2023-12-14T18:26:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f2961095700 1 -- start start 2023-12-14T18:26:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f2961095700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c101440 0x7f295c0ff090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.547 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f2961095700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c101d50 0x7f295c0ff5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f2961095700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f295c10a810 0x7f295c0ffb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f2961095700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c1a8e10 con 0x7f295c10a810 2023-12-14T18:26:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f2961095700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c19a5d0 con 0x7f295c101d50 2023-12-14T18:26:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f2961095700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c19a790 con 0x7f295c101440 2023-12-14T18:26:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f295b59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f295c10a810 0x7f295c0ffb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f295b59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f295c10a810 0x7f295c0ffb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46768/0 (socket says 172.21.15.93:46768) 2023-12-14T18:26:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f295b59e700 1 -- 172.21.15.93:0/2225371758 learned_addr learned my addr 172.21.15.93:0/2225371758 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f295ad9d700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c101440 0x7f295c0ff090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.548 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.542+0000 7f295a59c700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c101d50 0x7f295c0ff5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f295b59e700 1 -- 172.21.15.93:0/2225371758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c101440 msgr2=0x7f295c0ff090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f295b59e700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c101440 0x7f295c0ff090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f295b59e700 1 -- 172.21.15.93:0/2225371758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c101d50 msgr2=0x7f295c0ff5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f295b59e700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c101d50 0x7f295c0ff5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f295b59e700 1 -- 172.21.15.93:0/2225371758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2954009d00 con 0x7f295c10a810 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f295b59e700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f295c10a810 0x7f295c0ffb10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f295400b780 tx=0x7f295400bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f2943fff700 1 -- 172.21.15.93:0/2225371758 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f295400e040 con 0x7f295c10a810 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f2943fff700 1 -- 172.21.15.93:0/2225371758 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f29540094a0 con 0x7f295c10a810 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f2943fff700 1 -- 172.21.15.93:0/2225371758 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29540052a0 con 0x7f295c10a810 2023-12-14T18:26:56.549 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f2961095700 1 -- 172.21.15.93:0/2225371758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c1102e0 con 0x7f295c10a810 2023-12-14T18:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.543+0000 7f2961095700 1 -- 172.21.15.93:0/2225371758 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c110150 con 0x7f295c10a810 2023-12-14T18:26:56.552 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.545+0000 7f2943fff700 1 -- 172.21.15.93:0/2225371758 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 23) v1 ==== 135891+0+0 (secure 0 0 0) 0x7f295401d470 con 0x7f295c10a810 2023-12-14T18:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.546+0000 7f2943fff700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f29440a13d0 0x7f29440a3880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.546+0000 7f2943fff700 1 -- 172.21.15.93:0/2225371758 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f29540ce6f0 con 0x7f295c10a810 2023-12-14T18:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.547+0000 7f295ad9d700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f29440a13d0 0x7f29440a3880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.547+0000 7f295ad9d700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f29440a13d0 0x7f29440a3880 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f294c009c60 tx=0x7f294c009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f296d223b80 1 Processor -- start 2023-12-14T18:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f296d223b80 1 -- start start 2023-12-14T18:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f296d223b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f45e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.553 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f296d223b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f467e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f296d223b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f296d223b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82e30580 con 0x558b82f62100 2023-12-14T18:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f296d223b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82e9f810 con 0x558b82da3430 2023-12-14T18:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f296d223b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82e98ca0 con 0x558b82f46370 2023-12-14T18:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f295ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f45e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f295b59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f295ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f45e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:41674/0 (socket says 172.21.15.93:41674) 2023-12-14T18:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f295ad9d700 1 -- 172.21.15.93:0/893183080 learned_addr learned my addr 172.21.15.93:0/893183080 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:56.554 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.548+0000 7f295a59c700 1 --2- 172.21.15.93:0/893183080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f467e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f295ad9d700 1 -- 172.21.15.93:0/893183080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 msgr2=0x558b82f467e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f295ad9d700 1 --2- 172.21.15.93:0/893183080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f467e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f295ad9d700 1 -- 172.21.15.93:0/893183080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 msgr2=0x558b82f644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f295ad9d700 1 --2- 172.21.15.93:0/893183080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f295ad9d700 1 -- 172.21.15.93:0/893183080 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b82d1a350 con 0x558b82da3430 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f295b59e700 1 --2- 172.21.15.93:0/893183080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f295ad9d700 1 --2- 172.21.15.93:0/893183080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f45e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f294c03e4e0 tx=0x7f294c014000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b112aa863caae134 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f29417fa700 1 -- 172.21.15.93:0/893183080 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c03ee00 con 0x558b82da3430 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f29417fa700 1 -- 172.21.15.93:0/893183080 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f294c043070 con 0x558b82da3430 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.549+0000 7f29417fa700 1 -- 172.21.15.93:0/893183080 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c00e770 con 0x558b82da3430 2023-12-14T18:26:56.555 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 -- 172.21.15.93:0/893183080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 msgr2=0x558b82f45e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 --2- 172.21.15.93:0/893183080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f45e30 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f294c03e4e0 tx=0x7f294c014000 comp rx=0 tx=0).stop 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 -- 172.21.15.93:0/893183080 shutdown_connections 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 --2- 172.21.15.93:0/893183080 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 --2- 172.21.15.93:0/893183080 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f45e30 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 --2- 172.21.15.93:0/893183080 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f467e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 -- 172.21.15.93:0/893183080 >> 172.21.15.93:0/893183080 conn(0x558b82f3c890 msgr2=0x558b82f42200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 -- 172.21.15.93:0/893183080 shutdown_connections 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 -- 172.21.15.93:0/893183080 wait complete. 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.550+0000 7f296d223b80 1 -- start start 2023-12-14T18:26:56.556 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f296d223b80 1 Processor -- start 2023-12-14T18:26:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f296d223b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82da3430 0x558b82f52c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f296d223b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f46370 0x558b82f531c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.559 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f296d223b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 0x558b82f4dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f296d223b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82e34f30 con 0x558b82f46370 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f296d223b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82e34bd0 con 0x558b82f62100 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f296d223b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82d1c640 con 0x558b82da3430 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f296d223b80 10 client.0 _ll_register_callbacks cb 0x558b82e588b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f295a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f46370 0x558b82f531c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f295a59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f46370 0x558b82f531c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46786/0 (socket says 172.21.15.93:46786) 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f295a59c700 1 -- 172.21.15.93:0/2929334413 learned_addr learned my addr 172.21.15.93:0/2929334413 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.551+0000 7f295ad9d700 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82da3430 0x558b82f52c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f295b59e700 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 0x558b82f4dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f295a59c700 1 -- 172.21.15.93:0/2929334413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82da3430 msgr2=0x558b82f52c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:56.560 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f295a59c700 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82da3430 0x558b82f52c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f295a59c700 1 -- 172.21.15.93:0/2929334413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 msgr2=0x558b82f4dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f295a59c700 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 0x558b82f4dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f295a59c700 1 -- 172.21.15.93:0/2929334413 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f294c017070 con 0x558b82f46370 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f295a59c700 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f46370 0x558b82f531c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29500115e0 tx=0x7f29500118f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382d5aad7283bb93 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f293eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f293eff5700 1 -- 172.21.15.93:0/2929334413 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2950014650 con 0x558b82f46370 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f293eff5700 1 -- 172.21.15.93:0/2929334413 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f29500147b0 con 0x558b82f46370 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f296d223b80 1 -- 172.21.15.93:0/2929334413 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558b82f39ea0 con 0x558b82f46370 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f293eff5700 1 -- 172.21.15.93:0/2929334413 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f29500149b0 con 0x558b82f46370 2023-12-14T18:26:56.561 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f296d223b80 10 client.15567 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.552+0000 7f296d223b80 1 -- 172.21.15.93:0/2929334413 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558b82e5f110 con 0x558b82f46370 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.553+0000 7f293eff5700 1 -- 172.21.15.93:0/2929334413 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x558b82f39ea0 con 0x558b82f46370 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.553+0000 7f293eff5700 1 -- 172.21.15.93:0/2929334413 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f295001b020 con 0x558b82f46370 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.553+0000 7f296d223b80 10 client.15567 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.553+0000 7f296d223b80 10 client.15567 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.553+0000 7f296d223b80 20 client.15567 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.553+0000 7f296d223b80 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558b82f5e0e0 0x558b82f604f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.553+0000 7f296d223b80 4 client.15567 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.553+0000 7f296d223b80 1 -- 172.21.15.93:0/2929334413 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558b82e91c80 con 0x558b82f5e0e0 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.554+0000 7f295ad9d700 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558b82f5e0e0 0x558b82f604f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.562 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.554+0000 7f293eff5700 10 client.15567 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.554+0000 7f295ad9d700 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558b82f5e0e0 0x558b82f604f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a59df7b3b56d2b75 server_cookie=773e4bfad8e52a1d in_seq=0 out_seq=0 2023-12-14T18:26:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f293eff5700 1 -- 172.21.15.93:0/2929334413 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x558b82e91c80 con 0x558b82f5e0e0 2023-12-14T18:26:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f293eff5700 10 client.15567 handle_command_reply: tid=0 2023-12-14T18:26:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 client.15567 shutdown 2023-12-14T18:26:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 -- 172.21.15.93:0/2929334413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f46370 msgr2=0x558b82f531c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f46370 0x558b82f531c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f29500115e0 tx=0x7f29500118f0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 -- 172.21.15.93:0/2929334413 shutdown_connections 2023-12-14T18:26:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558b82f5e0e0 0x558b82f604f0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.563 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 0x558b82f4dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82da3430 0x558b82f52c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 --2- 172.21.15.93:0/2929334413 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f46370 0x558b82f531c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 -- 172.21.15.93:0/2929334413 >> 172.21.15.93:0/2929334413 conn(0x558b82f3c890 msgr2=0x558b82f41d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f293eff5700 0 client.15567 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 -- 172.21.15.93:0/2929334413 shutdown_connections 2023-12-14T18:26:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 1 -- 172.21.15.93:0/2929334413 wait complete. 2023-12-14T18:26:56.564 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.555+0000 7f296d223b80 20 client.15567 trim_cache size 0 max 16384 2023-12-14T18:26:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f296d223b80 1 Processor -- start 2023-12-14T18:26:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f296d223b80 1 -- start start 2023-12-14T18:26:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f296d223b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82da3430 0x558b82f604c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f296d223b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f57950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f296d223b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 0x558b82f46740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f296d223b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82f4e5f0 con 0x558b82da3430 2023-12-14T18:26:56.574 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f296d223b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82f49e40 con 0x558b82f62100 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f296d223b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82f644e0 con 0x558b82f46370 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f295ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82da3430 0x558b82f604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f295ad9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82da3430 0x558b82f604c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46800/0 (socket says 172.21.15.93:46800) 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f295ad9d700 1 -- 172.21.15.93:0/801846815 learned_addr learned my addr 172.21.15.93:0/801846815 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f295a59c700 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f57950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f295ad9d700 1 -- 172.21.15.93:0/801846815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 msgr2=0x558b82f57950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f295ad9d700 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f57950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f295b59e700 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 0x558b82f46740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.570+0000 7f295ad9d700 1 -- 172.21.15.93:0/801846815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 msgr2=0x558b82f46740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.571+0000 7f295ad9d700 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 0x558b82f46740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.575 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.571+0000 7f295ad9d700 1 -- 172.21.15.93:0/801846815 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558b82e5ee60 con 0x558b82da3430 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.571+0000 7f295a59c700 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.571+0000 7f295b59e700 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 0x558b82f46740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.571+0000 7f295ad9d700 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82da3430 0x558b82f604c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f294c03e4b0 tx=0x7f294c005ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cec37065c70ecc2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.571+0000 7f2941ffb700 1 -- 172.21.15.93:0/801846815 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c04f040 con 0x558b82da3430 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.571+0000 7f2941ffb700 1 -- 172.21.15.93:0/801846815 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f294c053da0 con 0x558b82da3430 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.571+0000 7f2941ffb700 1 -- 172.21.15.93:0/801846815 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c053780 con 0x558b82da3430 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 -- 172.21.15.93:0/801846815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82da3430 msgr2=0x558b82f604c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82da3430 0x558b82f604c0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f294c03e4b0 tx=0x7f294c005ce0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 -- 172.21.15.93:0/801846815 shutdown_connections 2023-12-14T18:26:56.576 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82da3430 0x558b82f604c0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82f62100 0x558b82f46740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 --2- 172.21.15.93:0/801846815 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f57950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 -- 172.21.15.93:0/801846815 >> 172.21.15.93:0/801846815 conn(0x558b82f3c890 msgr2=0x558b82f68aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 -- 172.21.15.93:0/801846815 shutdown_connections 2023-12-14T18:26:56.577 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 -- 172.21.15.93:0/801846815 wait complete. 2023-12-14T18:26:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 -- start start 2023-12-14T18:26:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.572+0000 7f296d223b80 1 Processor -- start 2023-12-14T18:26:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f296d223b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f53ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f296d223b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f54420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f296d223b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f54960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f296d223b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82fb3cf0 con 0x558b82f62100 2023-12-14T18:26:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f296d223b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82fb3e60 con 0x558b82da3430 2023-12-14T18:26:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f296d223b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x558b82fb4020 con 0x558b82f46370 2023-12-14T18:26:56.580 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f296d223b80 10 client.0 _ll_register_callbacks cb 0x7f2950009b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f295b59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f54960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f295ad9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f53ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f295b59e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f54960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46814/0 (socket says 172.21.15.93:46814) 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f295b59e700 1 -- 172.21.15.93:0/2984491731 learned_addr learned my addr 172.21.15.93:0/2984491731 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.573+0000 7f295a59c700 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f54420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f295ad9d700 1 -- 172.21.15.93:0/2984491731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 msgr2=0x558b82f54420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f295ad9d700 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f54420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f295ad9d700 1 -- 172.21.15.93:0/2984491731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 msgr2=0x558b82f54960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f295ad9d700 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f54960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.581 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f295ad9d700 1 -- 172.21.15.93:0/2984491731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558b82d1a350 con 0x558b82da3430 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f295ad9d700 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f53ee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f294c011ab0 tx=0x7f294c0051c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=256a74bfca846788 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f293eff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f293eff5700 1 -- 172.21.15.93:0/2984491731 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c04f040 con 0x558b82da3430 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f296d223b80 1 -- 172.21.15.93:0/2984491731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f294c043070 con 0x558b82da3430 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f293eff5700 1 -- 172.21.15.93:0/2984491731 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f294c045070 con 0x558b82da3430 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f293eff5700 1 -- 172.21.15.93:0/2984491731 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f294c005510 con 0x558b82da3430 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f296d223b80 10 client.25211 fetch_fsmap learned FSMap version 24 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f296d223b80 1 -- 172.21.15.93:0/2984491731 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x558b82e45940 con 0x558b82da3430 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.574+0000 7f293eff5700 1 -- 172.21.15.93:0/2984491731 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f294c043070 con 0x558b82da3430 2023-12-14T18:26:56.582 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.575+0000 7f293eff5700 1 -- 172.21.15.93:0/2984491731 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f294c052070 con 0x558b82da3430 2023-12-14T18:26:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.575+0000 7f296d223b80 10 client.25211 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:26:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.575+0000 7f296d223b80 10 client.25211 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:26:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.575+0000 7f296d223b80 20 client.25211 populate_metadata read hostname 'smithi093' 2023-12-14T18:26:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.575+0000 7f296d223b80 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558b82fb5370 0x558b82fb7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:26:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.575+0000 7f296d223b80 4 client.25211 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:26:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.575+0000 7f296d223b80 1 -- 172.21.15.93:0/2984491731 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x558b82e919a0 con 0x558b82fb5370 2023-12-14T18:26:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.575+0000 7f295a59c700 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558b82fb5370 0x558b82fb7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:26:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.576+0000 7f293eff5700 10 client.25211 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:56.583 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.576+0000 7f295a59c700 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558b82fb5370 0x558b82fb7780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50b5f4f470f40a9 server_cookie=2b2d10f58ab5370d in_seq=0 out_seq=0 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.579+0000 7f293eff5700 1 -- 172.21.15.93:0/2984491731 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x558b82e919a0 con 0x558b82fb5370 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.579+0000 7f293eff5700 10 client.25211 handle_command_reply: tid=0 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.579+0000 7f296d223b80 1 client.25211 shutdown 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 -- 172.21.15.93:0/2984491731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 msgr2=0x558b82f53ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f53ee0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f294c011ab0 tx=0x7f294c0051c0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 -- 172.21.15.93:0/2984491731 shutdown_connections 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x558b82fb5370 0x558b82fb7780 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x558b82f62100 0x558b82f54960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f293eff5700 0 client.25211 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:26:56.585 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x558b82da3430 0x558b82f53ee0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 --2- 172.21.15.93:0/2984491731 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x558b82f46370 0x558b82f54420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 -- 172.21.15.93:0/2984491731 >> 172.21.15.93:0/2984491731 conn(0x558b82f3c890 msgr2=0x558b82f46740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 -- 172.21.15.93:0/2984491731 shutdown_connections 2023-12-14T18:26:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 1 -- 172.21.15.93:0/2984491731 wait complete. 2023-12-14T18:26:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f296d223b80 20 client.25211 trim_cache size 0 max 16384 2023-12-14T18:26:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:26:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:26:56.586 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:26:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:26:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 -- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f29440a13d0 msgr2=0x7f29440a3880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.587 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f29440a13d0 0x7f29440a3880 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f294c009c60 tx=0x7f294c009260 comp rx=0 tx=0).stop 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 -- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f295c10a810 msgr2=0x7f295c0ffb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f295c10a810 0x7f295c0ffb10 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f295400b780 tx=0x7f295400bd60 comp rx=0 tx=0).stop 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 -- 172.21.15.93:0/2225371758 shutdown_connections 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f29440a13d0 0x7f29440a3880 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f295c10a810 0x7f295c0ffb10 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f295c101440 0x7f295c0ff090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 --2- 172.21.15.93:0/2225371758 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f295c101d50 0x7f295c0ff5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.580+0000 7f2961095700 1 -- 172.21.15.93:0/2225371758 >> 172.21.15.93:0/2225371758 conn(0x7f295c0763a0 msgr2=0x7f295c104290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.583+0000 7f2961095700 1 -- 172.21.15.93:0/2225371758 shutdown_connections 2023-12-14T18:26:56.588 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:26:56.583+0000 7f2961095700 1 -- 172.21.15.93:0/2225371758 wait complete. 2023-12-14T18:26:57.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:56 smithi097 ceph-mon[43486]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-12-14T18:26:57.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:56 smithi160 ceph-mon[43840]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-12-14T18:26:57.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:56 smithi093 ceph-mon[42070]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.5 KiB/s wr, 12 op/s 2023-12-14T18:26:59.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:26:58 smithi097 ceph-mon[43486]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-12-14T18:26:59.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:26:58 smithi160 ceph-mon[43840]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-12-14T18:26:59.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:26:58 smithi093 ceph-mon[42070]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 5 op/s 2023-12-14T18:27:00.815 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:27:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:27:00.315+0000 7fc5792de700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:00.815 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:27:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:27:00.315+0000 7fc5792de700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:00.815 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:27:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:27:00.315+0000 7fc5792de700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:00.815 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:27:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:27:00.315+0000 7fc5792de700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:00.816 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:27:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:27:00.315+0000 7fc5792de700 -1 client.0 error registering admin socket command: (17) File exists 2023-12-14T18:27:01.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:00 smithi097 ceph-mon[43486]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-14T18:27:01.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:00 smithi160 ceph-mon[43840]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-14T18:27:01.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:00 smithi093 ceph-mon[42070]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 7 op/s 2023-12-14T18:27:03.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:02 smithi160 ceph-mon[43840]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-14T18:27:03.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:02 smithi093 ceph-mon[42070]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-14T18:27:03.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:02 smithi097 ceph-mon[43486]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 255 B/s wr, 5 op/s 2023-12-14T18:27:04.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:03 smithi160 ceph-mon[43840]: mgrmap e24: x(active, since 35s), standbys: y, z 2023-12-14T18:27:04.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:03 smithi093 ceph-mon[42070]: mgrmap e24: x(active, since 35s), standbys: y, z 2023-12-14T18:27:04.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:03 smithi097 ceph-mon[43486]: mgrmap e24: x(active, since 35s), standbys: y, z 2023-12-14T18:27:05.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:04 smithi160 ceph-mon[43840]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-12-14T18:27:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:04 smithi093 ceph-mon[42070]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-12-14T18:27:05.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:04 smithi097 ceph-mon[43486]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2023-12-14T18:27:07.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:06 smithi160 ceph-mon[43840]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2023-12-14T18:27:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:06 smithi093 ceph-mon[42070]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2023-12-14T18:27:07.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:06 smithi097 ceph-mon[43486]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 19 op/s 2023-12-14T18:27:09.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:08 smithi160 ceph-mon[43840]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-12-14T18:27:09.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:08 smithi093 ceph-mon[42070]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-12-14T18:27:09.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:08 smithi097 ceph-mon[43486]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-12-14T18:27:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 -- 172.21.15.93:0/3522532893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ffd60 msgr2=0x7f668c103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 --2- 172.21.15.93:0/3522532893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ffd60 0x7f668c103d20 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f66800099c0 tx=0x7f6680009cd0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 -- 172.21.15.93:0/3522532893 shutdown_connections 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 --2- 172.21.15.93:0/3522532893 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c104400 0x7f668c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 --2- 172.21.15.93:0/3522532893 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f668c0ff3c0 0x7f668c0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 --2- 172.21.15.93:0/3522532893 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ffd60 0x7f668c103d20 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 -- 172.21.15.93:0/3522532893 >> 172.21.15.93:0/3522532893 conn(0x7f668c0763a0 msgr2=0x7f668c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 -- 172.21.15.93:0/3522532893 shutdown_connections 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 -- 172.21.15.93:0/3522532893 wait complete. 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.698+0000 7f66912f1700 1 Processor -- start 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f66912f1700 1 -- start start 2023-12-14T18:27:09.703 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f66912f1700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ff3c0 0x7f668c195340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f66912f1700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0ffd60 0x7f668c195880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f66912f1700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f668c104400 0x7f668c199ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f66912f1700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c19a320 con 0x7f668c0ff3c0 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f66912f1700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c19a490 con 0x7f668c0ffd60 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f66912f1700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f668c19a650 con 0x7f668c104400 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ff3c0 0x7f668c195340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ff3c0 0x7f668c195340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46274/0 (socket says 172.21.15.93:46274) 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668affd700 1 -- 172.21.15.93:0/480580378 learned_addr learned my addr 172.21.15.93:0/480580378 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668b7fe700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f668c104400 0x7f668c199ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668a7fc700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0ffd60 0x7f668c195880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668affd700 1 -- 172.21.15.93:0/480580378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f668c104400 msgr2=0x7f668c199ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668affd700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f668c104400 0x7f668c199ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668affd700 1 -- 172.21.15.93:0/480580378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0ffd60 msgr2=0x7f668c195880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668affd700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0ffd60 0x7f668c195880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668affd700 1 -- 172.21.15.93:0/480580378 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66800095f0 con 0x7f668c0ff3c0 2023-12-14T18:27:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f668affd700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ff3c0 0x7f668c195340 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f667c00e9a0 tx=0x7f667c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.699+0000 7f6673fff700 1 -- 172.21.15.93:0/480580378 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c009790 con 0x7f668c0ff3c0 2023-12-14T18:27:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.700+0000 7f6673fff700 1 -- 172.21.15.93:0/480580378 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f667c010430 con 0x7f668c0ff3c0 2023-12-14T18:27:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.700+0000 7f66912f1700 1 -- 172.21.15.93:0/480580378 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f668c10f180 con 0x7f668c0ff3c0 2023-12-14T18:27:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.700+0000 7f6673fff700 1 -- 172.21.15.93:0/480580378 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f667c018680 con 0x7f668c0ff3c0 2023-12-14T18:27:09.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.700+0000 7f66912f1700 1 -- 172.21.15.93:0/480580378 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c10eff0 con 0x7f668c0ff3c0 2023-12-14T18:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.701+0000 7f6673fff700 1 -- 172.21.15.93:0/480580378 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f667c0105a0 con 0x7f668c0ff3c0 2023-12-14T18:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.702+0000 7f6673fff700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f66740a13e0 0x7f66740a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.702+0000 7f668a7fc700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f66740a13e0 0x7f66740a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.702+0000 7f6673fff700 1 -- 172.21.15.93:0/480580378 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f667c014070 con 0x7f668c0ff3c0 2023-12-14T18:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f668a7fc700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f66740a13e0 0x7f66740a3890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f66800098e0 tx=0x7f6680000bc0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f669d47fb80 1 Processor -- start 2023-12-14T18:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f669d47fb80 1 -- start start 2023-12-14T18:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 0x55ffd5986e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd5987370 0x55ffd59877e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd59944d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f669d47fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5871580 con 0x55ffd59920f0 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f669d47fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd58e0810 con 0x55ffd5987370 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f669d47fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd58d9ca0 con 0x55ffd57e4430 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f668b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd59944d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f668b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd59944d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46288/0 (socket says 172.21.15.93:46288) 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f668b7fe700 1 -- 172.21.15.93:0/1498681740 learned_addr learned my addr 172.21.15.93:0/1498681740 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.703+0000 7f668affd700 1 --2- 172.21.15.93:0/1498681740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 0x55ffd5986e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f668a7fc700 1 --2- 172.21.15.93:0/1498681740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd5987370 0x55ffd59877e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f668b7fe700 1 -- 172.21.15.93:0/1498681740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 msgr2=0x55ffd5986e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f668b7fe700 1 --2- 172.21.15.93:0/1498681740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 0x55ffd5986e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f668b7fe700 1 -- 172.21.15.93:0/1498681740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd5987370 msgr2=0x55ffd59877e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f668b7fe700 1 --2- 172.21.15.93:0/1498681740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd5987370 0x55ffd59877e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f668b7fe700 1 -- 172.21.15.93:0/1498681740 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffd575b350 con 0x55ffd59920f0 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f668b7fe700 1 --2- 172.21.15.93:0/1498681740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd59944d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6684003fa0 tx=0x7f6684009ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c868a9e780a5ca server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f66717fa700 1 -- 172.21.15.93:0/1498681740 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684019070 con 0x55ffd59920f0 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f66717fa700 1 -- 172.21.15.93:0/1498681740 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f668400f040 con 0x55ffd59920f0 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.704+0000 7f66717fa700 1 -- 172.21.15.93:0/1498681740 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684007a40 con 0x55ffd59920f0 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 -- 172.21.15.93:0/1498681740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 msgr2=0x55ffd59944d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1498681740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd59944d0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f6684003fa0 tx=0x7f6684009ba0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 -- 172.21.15.93:0/1498681740 shutdown_connections 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1498681740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd59944d0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1498681740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 0x55ffd5986e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1498681740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd5987370 0x55ffd59877e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 -- 172.21.15.93:0/1498681740 >> 172.21.15.93:0/1498681740 conn(0x55ffd597d890 msgr2=0x55ffd5983200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 -- 172.21.15.93:0/1498681740 shutdown_connections 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 -- 172.21.15.93:0/1498681740 wait complete. 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 -- start start 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.705+0000 7f669d47fb80 1 Processor -- start 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 0x55ffd59919f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd5987370 0x55ffd598ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd598f290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f669d47fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5875f30 con 0x55ffd59920f0 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f669d47fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5875bd0 con 0x55ffd57e4430 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f669d47fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd575d640 con 0x55ffd5987370 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f669d47fb80 10 client.0 _ll_register_callbacks cb 0x55ffd58998b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:09.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd598f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd598f290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46292/0 (socket says 172.21.15.93:46292) 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668b7fe700 1 -- 172.21.15.93:0/1570402741 learned_addr learned my addr 172.21.15.93:0/1570402741 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668affd700 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 0x55ffd59919f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668a7fc700 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd5987370 0x55ffd598ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668b7fe700 1 -- 172.21.15.93:0/1570402741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd5987370 msgr2=0x55ffd598ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668b7fe700 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd5987370 0x55ffd598ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668b7fe700 1 -- 172.21.15.93:0/1570402741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 msgr2=0x55ffd59919f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668b7fe700 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 0x55ffd59919f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668b7fe700 1 -- 172.21.15.93:0/1570402741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6684003bd0 con 0x55ffd59920f0 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f668b7fe700 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd598f290 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6684000f80 tx=0x7f6684004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ecd41cf1002a624 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.717 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f666eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f666eff5700 1 -- 172.21.15.93:0/1570402741 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6684019040 con 0x55ffd59920f0 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f666eff5700 1 -- 172.21.15.93:0/1570402741 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f6684007af0 con 0x55ffd59920f0 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.706+0000 7f669d47fb80 1 -- 172.21.15.93:0/1570402741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffd597aea0 con 0x55ffd59920f0 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f666eff5700 1 -- 172.21.15.93:0/1570402741 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6684007cb0 con 0x55ffd59920f0 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f669d47fb80 10 client.15591 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f669d47fb80 1 -- 172.21.15.93:0/1570402741 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ffd58a0110 con 0x55ffd59920f0 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f666eff5700 1 -- 172.21.15.93:0/1570402741 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffd597aea0 con 0x55ffd59920f0 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f666eff5700 1 -- 172.21.15.93:0/1570402741 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f668401f450 con 0x55ffd59920f0 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f669d47fb80 10 client.15591 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f669d47fb80 10 client.15591 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f669d47fb80 20 client.15591 populate_metadata read hostname 'smithi093' 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ffd59a1a70 0x55ffd59a3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f669d47fb80 4 client.15591 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:09.718 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f669d47fb80 1 -- 172.21.15.93:0/1570402741 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ffd58d2c80 con 0x55ffd59a1a70 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.707+0000 7f668affd700 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ffd59a1a70 0x55ffd59a3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f668affd700 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ffd59a1a70 0x55ffd59a3e80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e26f0218422c9d56 server_cookie=f4ff40b5efe9a40d in_seq=0 out_seq=0 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f666eff5700 10 client.15591 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f666eff5700 1 -- 172.21.15.93:0/1570402741 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ffd58d2c80 con 0x55ffd59a1a70 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f666eff5700 10 client.15591 handle_command_reply: tid=0 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 client.15591 shutdown 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 -- 172.21.15.93:0/1570402741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 msgr2=0x55ffd598f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd598f290 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f6684000f80 tx=0x7f6684004750 comp rx=0 tx=0).stop 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 -- 172.21.15.93:0/1570402741 shutdown_connections 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ffd59a1a70 0x55ffd59a3e80 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd59920f0 0x55ffd598f290 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 0x55ffd59919f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.719 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f666eff5700 0 client.15591 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1570402741 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd5987370 0x55ffd598ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 -- 172.21.15.93:0/1570402741 >> 172.21.15.93:0/1570402741 conn(0x55ffd597d890 msgr2=0x55ffd5982df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 -- 172.21.15.93:0/1570402741 shutdown_connections 2023-12-14T18:27:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.708+0000 7f669d47fb80 1 -- 172.21.15.93:0/1570402741 wait complete. 2023-12-14T18:27:09.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.709+0000 7f669d47fb80 20 client.15591 trim_cache size 0 max 16384 2023-12-14T18:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.723+0000 7f669d47fb80 1 Processor -- start 2023-12-14T18:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.723+0000 7f669d47fb80 1 -- start start 2023-12-14T18:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.723+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 0x55ffd5998cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.723+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd598a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.723+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd59920f0 0x55ffd598a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.731 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.723+0000 7f669d47fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd598ab60 con 0x55ffd5987370 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.723+0000 7f669d47fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd5994a10 con 0x55ffd59920f0 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.723+0000 7f669d47fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd59944d0 con 0x55ffd57e4430 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd598a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd598a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46300/0 (socket says 172.21.15.93:46300) 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668a7fc700 1 -- 172.21.15.93:0/1766202987 learned_addr learned my addr 172.21.15.93:0/1766202987 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668affd700 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 0x55ffd5998cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668b7fe700 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd59920f0 0x55ffd598a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668a7fc700 1 -- 172.21.15.93:0/1766202987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 msgr2=0x55ffd5998cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668a7fc700 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 0x55ffd5998cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668a7fc700 1 -- 172.21.15.93:0/1766202987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd59920f0 msgr2=0x55ffd598a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668a7fc700 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd59920f0 0x55ffd598a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668a7fc700 1 -- 172.21.15.93:0/1766202987 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ffd589fe60 con 0x55ffd5987370 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668b7fe700 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd59920f0 0x55ffd598a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:09.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f668a7fc700 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd598a020 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66800036c0 tx=0x7f6680022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bbeb37fdafc2a2d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f6671ffb700 1 -- 172.21.15.93:0/1766202987 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000bd70 con 0x55ffd5987370 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f6671ffb700 1 -- 172.21.15.93:0/1766202987 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f668002d040 con 0x55ffd5987370 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.724+0000 7f6671ffb700 1 -- 172.21.15.93:0/1766202987 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f668000f750 con 0x55ffd5987370 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 -- 172.21.15.93:0/1766202987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 msgr2=0x55ffd598a020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd598a020 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f66800036c0 tx=0x7f6680022000 comp rx=0 tx=0).stop 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 -- 172.21.15.93:0/1766202987 shutdown_connections 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd59920f0 0x55ffd598a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd57e4430 0x55ffd5998cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1766202987 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd598a020 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 -- 172.21.15.93:0/1766202987 >> 172.21.15.93:0/1766202987 conn(0x55ffd597d890 msgr2=0x55ffd599de00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.733 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 -- 172.21.15.93:0/1766202987 shutdown_connections 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 -- 172.21.15.93:0/1766202987 wait complete. 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 -- start start 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.725+0000 7f669d47fb80 1 Processor -- start 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 0x55ffd59f34c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd59f3a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f669d47fb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd59920f0 0x55ffd59edc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f669d47fb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd59ee270 con 0x55ffd5987370 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f669d47fb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd59ee3e0 con 0x55ffd57e4430 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f669d47fb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ffd59ee5a0 con 0x55ffd59920f0 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f669d47fb80 10 client.0 _ll_register_callbacks cb 0x7f6684003df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd59f3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.734 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 0x55ffd59f34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd59f3a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46308/0 (socket says 172.21.15.93:46308) 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668a7fc700 1 -- 172.21.15.93:0/1545941803 learned_addr learned my addr 172.21.15.93:0/1545941803 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668b7fe700 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd59920f0 0x55ffd59edc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668a7fc700 1 -- 172.21.15.93:0/1545941803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd59920f0 msgr2=0x55ffd59edc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668a7fc700 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd59920f0 0x55ffd59edc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668a7fc700 1 -- 172.21.15.93:0/1545941803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 msgr2=0x55ffd59f34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668a7fc700 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 0x55ffd59f34c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668a7fc700 1 -- 172.21.15.93:0/1545941803 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f668c10c810 con 0x55ffd5987370 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668affd700 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 0x55ffd59f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f668a7fc700 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd59f3a00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6680034040 tx=0x7f6680004180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a43bf6c028a5848 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f666eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f666eff5700 1 -- 172.21.15.93:0/1545941803 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6680004600 con 0x55ffd5987370 2023-12-14T18:27:09.735 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f666eff5700 1 -- 172.21.15.93:0/1545941803 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f668000f660 con 0x55ffd5987370 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.726+0000 7f669d47fb80 1 -- 172.21.15.93:0/1545941803 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ffd5987200 con 0x55ffd5987370 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f666eff5700 1 -- 172.21.15.93:0/1545941803 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6680003a20 con 0x55ffd5987370 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f669d47fb80 10 client.15597 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f669d47fb80 1 -- 172.21.15.93:0/1545941803 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55ffd5886940 con 0x55ffd5987370 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f666eff5700 1 -- 172.21.15.93:0/1545941803 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ffd5987200 con 0x55ffd5987370 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f666eff5700 1 -- 172.21.15.93:0/1545941803 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f66800266e0 con 0x55ffd5987370 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f669d47fb80 10 client.15597 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f669d47fb80 10 client.15597 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f669d47fb80 20 client.15597 populate_metadata read hostname 'smithi093' 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ffd59faef0 0x55ffd59fd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f669d47fb80 4 client.15597 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f669d47fb80 1 -- 172.21.15.93:0/1545941803 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f668400f040 con 0x55ffd59faef0 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.727+0000 7f668affd700 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ffd59faef0 0x55ffd59fd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:09.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.728+0000 7f666eff5700 10 client.15597 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:09.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.728+0000 7f668affd700 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ffd59faef0 0x55ffd59fd300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1abb4e5a133d1116 server_cookie=fb03593ac5a4cfab in_seq=0 out_seq=0 2023-12-14T18:27:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f666eff5700 1 -- 172.21.15.93:0/1545941803 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f668400f040 con 0x55ffd59faef0 2023-12-14T18:27:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f666eff5700 10 client.15597 handle_command_reply: tid=0 2023-12-14T18:27:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 client.15597 shutdown 2023-12-14T18:27:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 -- 172.21.15.93:0/1545941803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 msgr2=0x55ffd59f3a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd59f3a00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f6680034040 tx=0x7f6680004180 comp rx=0 tx=0).stop 2023-12-14T18:27:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 -- 172.21.15.93:0/1545941803 shutdown_connections 2023-12-14T18:27:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ffd59faef0 0x55ffd59fd300 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.738 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ffd59920f0 0x55ffd59edc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ffd57e4430 0x55ffd59f34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 --2- 172.21.15.93:0/1545941803 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ffd5987370 0x55ffd59f3a00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f666eff5700 0 client.15597 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 -- 172.21.15.93:0/1545941803 >> 172.21.15.93:0/1545941803 conn(0x55ffd597d890 msgr2=0x55ffd5988f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 -- 172.21.15.93:0/1545941803 shutdown_connections 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.735+0000 7f669d47fb80 1 -- 172.21.15.93:0/1545941803 wait complete. 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f669d47fb80 20 client.15597 trim_cache size 0 max 16384 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:27:09.739 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:27:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:27:09.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 -- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f66740a13e0 msgr2=0x7f66740a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f66740a13e0 0x7f66740a3890 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f66800098e0 tx=0x7f6680000bc0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 -- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ff3c0 msgr2=0x7f668c195340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ff3c0 0x7f668c195340 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f667c00e9a0 tx=0x7f667c00ecb0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 -- 172.21.15.93:0/480580378 shutdown_connections 2023-12-14T18:27:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f66740a13e0 0x7f66740a3890 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f668c0ff3c0 0x7f668c195340 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f668c104400 0x7f668c199ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 --2- 172.21.15.93:0/480580378 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f668c0ffd60 0x7f668c195880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 -- 172.21.15.93:0/480580378 >> 172.21.15.93:0/480580378 conn(0x7f668c0763a0 msgr2=0x7f668c0fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 -- 172.21.15.93:0/480580378 shutdown_connections 2023-12-14T18:27:09.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:09.736+0000 7f66912f1700 1 -- 172.21.15.93:0/480580378 wait complete. 2023-12-14T18:27:11.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:10 smithi160 ceph-mon[43840]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2023-12-14T18:27:11.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:10 smithi093 ceph-mon[42070]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2023-12-14T18:27:11.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:10 smithi097 ceph-mon[43486]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 24 KiB/s wr, 21 op/s 2023-12-14T18:27:12.187 INFO:tasks.workunit.client.0.smithi093.stderr:Note: switching to 'bf026da9d69000f3d4a9d16360156095d6612a2d'. 2023-12-14T18:27:12.187 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-12-14T18:27:12.187 INFO:tasks.workunit.client.0.smithi093.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2023-12-14T18:27:12.187 INFO:tasks.workunit.client.0.smithi093.stderr:changes and commit them, and you can discard any commits you make in this 2023-12-14T18:27:12.187 INFO:tasks.workunit.client.0.smithi093.stderr:state without impacting any branches by switching back to a branch. 2023-12-14T18:27:12.187 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-12-14T18:27:12.187 INFO:tasks.workunit.client.0.smithi093.stderr:If you want to create a new branch to retain commits you create, you may 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr:do so (now or later) by using -c with the switch command. Example: 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr: git switch -c 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr:Or undo this operation with: 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr: git switch - 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2023-12-14T18:27:12.188 INFO:tasks.workunit.client.0.smithi093.stderr: 2023-12-14T18:27:12.189 INFO:tasks.workunit.client.0.smithi093.stderr:HEAD is now at bf026da9d69 doc/cephfs: Update docs to match remove functionality and respective vxattrs 2023-12-14T18:27:12.190 DEBUG:teuthology.orchestra.run.smithi093:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2023-12-14T18:27:12.259 INFO:tasks.workunit.client.0.smithi093.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2023-12-14T18:27:12.261 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-14T18:27:12.261 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2023-12-14T18:27:12.784 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2023-12-14T18:27:12.839 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2023-12-14T18:27:12.878 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2023-12-14T18:27:12.880 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-14T18:27:12.880 INFO:tasks.workunit.client.0.smithi093.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2023-12-14T18:27:12.916 INFO:tasks.workunit.client.0.smithi093.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2023-12-14T18:27:12.919 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:27:12.919 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2023-12-14T18:27:12.977 INFO:tasks.workunit:Running workunits matching fs/misc on client.0... 2023-12-14T18:27:12.979 INFO:tasks.workunit:Running workunit fs/misc/acl.sh... 2023-12-14T18:27:12.979 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/acl.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/acl.sh 2023-12-14T18:27:13.040 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-12-14T18:27:13.040 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir -p testdir 2023-12-14T18:27:13.042 INFO:tasks.workunit.client.0.smithi093.stderr:+ cd testdir 2023-12-14T18:27:13.042 INFO:tasks.workunit.client.0.smithi093.stderr:+ set +e 2023-12-14T18:27:13.042 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfacl -d -m u:nobody:rw . 2023-12-14T18:27:13.061 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-12-14T18:27:13.061 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-12-14T18:27:13.061 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=0 2023-12-14T18:27:13.061 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 0 -lt 100 ']' 2023-12-14T18:27:13.062 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 0 + 1 2023-12-14T18:27:13.062 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=1 2023-12-14T18:27:13.062 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:13.064 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:13.064 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:13.069 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:13.069 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:13.069 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:13.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:12 smithi160 ceph-mon[43840]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-12-14T18:27:13.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:12 smithi093 ceph-mon[42070]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-12-14T18:27:13.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:12 smithi097 ceph-mon[43486]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 81 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 24 KiB/s wr, 18 op/s 2023-12-14T18:27:13.849 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:13.849 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:14.045 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:14.045 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:14.081 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:14.081 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:14.081 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 1 -lt 100 ']' 2023-12-14T18:27:14.081 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 1 + 1 2023-12-14T18:27:14.155 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=2 2023-12-14T18:27:14.155 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:14.210 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:14.210 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:14.211 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:14.211 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:14.211 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:14.698 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:14.699 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:14.870 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:14.870 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:14.882 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:14.882 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:14.882 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -lt 100 ']' 2023-12-14T18:27:14.883 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 2 + 1 2023-12-14T18:27:14.905 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=3 2023-12-14T18:27:14.905 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:14.934 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:14.935 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:14.936 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:14.936 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:14.936 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:15.212 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:14 smithi093 ceph-mon[42070]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 20 op/s 2023-12-14T18:27:15.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:14 smithi160 ceph-mon[43840]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 20 op/s 2023-12-14T18:27:15.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:14 smithi097 ceph-mon[43486]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 26 KiB/s wr, 20 op/s 2023-12-14T18:27:15.398 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:15.399 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:15.745 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:15.745 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:15.763 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:15.763 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:15.763 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 3 -lt 100 ']' 2023-12-14T18:27:15.763 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 3 + 1 2023-12-14T18:27:15.788 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=4 2023-12-14T18:27:15.788 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:15.819 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:15.819 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:15.821 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:15.821 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:15.821 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:16.673 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:16.674 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:16.836 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:16.837 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:16.862 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:16.862 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:16.862 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 4 -lt 100 ']' 2023-12-14T18:27:16.862 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 4 + 1 2023-12-14T18:27:16.888 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=5 2023-12-14T18:27:16.888 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:16.916 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:16.916 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:16.917 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:16.917 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:16.917 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:17.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:16 smithi160 ceph-mon[43840]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 KiB/s wr, 20 op/s 2023-12-14T18:27:17.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:16 smithi093 ceph-mon[42070]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 KiB/s wr, 20 op/s 2023-12-14T18:27:17.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:16 smithi097 ceph-mon[43486]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 25 KiB/s wr, 20 op/s 2023-12-14T18:27:17.640 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:17.640 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:17.819 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:17.819 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:17.833 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:17.833 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:17.833 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 5 -lt 100 ']' 2023-12-14T18:27:17.833 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 5 + 1 2023-12-14T18:27:17.871 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=6 2023-12-14T18:27:17.871 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:17.946 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:17.947 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:17.947 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:17.947 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:17.947 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:18.993 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:18.993 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:19.202 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:19.203 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:19.217 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:19.217 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:19.217 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 6 -lt 100 ']' 2023-12-14T18:27:19.218 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 6 + 1 2023-12-14T18:27:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:18 smithi160 ceph-mon[43840]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-12-14T18:27:19.257 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=7 2023-12-14T18:27:19.257 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:19.275 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:19.275 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:19.276 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:19.277 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:19.277 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:18 smithi093 ceph-mon[42070]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-12-14T18:27:19.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:18 smithi097 ceph-mon[43486]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 KiB/s rd, 2.8 KiB/s wr, 7 op/s 2023-12-14T18:27:19.748 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:19.748 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:19.878 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:19.878 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:19.890 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:19.890 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:19.890 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 7 -lt 100 ']' 2023-12-14T18:27:19.890 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 7 + 1 2023-12-14T18:27:19.912 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=8 2023-12-14T18:27:19.913 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:19.936 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:19.936 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:19.937 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:19.937 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:19.937 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:20.339 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:20.340 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:20.461 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:20.461 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:20.474 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:20.474 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:20.474 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 8 -lt 100 ']' 2023-12-14T18:27:20.475 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 8 + 1 2023-12-14T18:27:20.515 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=9 2023-12-14T18:27:20.516 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:20.535 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:20.535 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:20.536 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:20.537 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:20.537 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:20.897 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:20.898 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:21.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:20 smithi093 ceph-mon[42070]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2023-12-14T18:27:21.086 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:21.086 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:21.138 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:21.138 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:21.138 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 9 -lt 100 ']' 2023-12-14T18:27:21.138 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 9 + 1 2023-12-14T18:27:21.174 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=10 2023-12-14T18:27:21.174 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:21.192 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:21.192 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:21.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:21.194 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:21.194 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:20 smithi160 ceph-mon[43840]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2023-12-14T18:27:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:20 smithi097 ceph-mon[43486]: pgmap v29: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 6.0 KiB/s wr, 10 op/s 2023-12-14T18:27:21.597 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:21.597 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:21.735 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:21.736 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:21.749 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:21.749 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:21.750 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 10 -lt 100 ']' 2023-12-14T18:27:21.750 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 10 + 1 2023-12-14T18:27:21.787 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=11 2023-12-14T18:27:21.787 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:21.815 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:21.815 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:21.816 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:21.816 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:21.817 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:22.180 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:22.181 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:22.319 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:22.319 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:22.332 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:22.332 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:22.332 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 11 -lt 100 ']' 2023-12-14T18:27:22.332 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 11 + 1 2023-12-14T18:27:22.362 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=12 2023-12-14T18:27:22.362 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:22.410 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:22.410 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:22.412 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:22.412 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:22.412 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:23.031 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:23.031 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:23.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:22 smithi093 ceph-mon[42070]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.9 KiB/s wr, 7 op/s 2023-12-14T18:27:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:22 smithi160 ceph-mon[43840]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.9 KiB/s wr, 7 op/s 2023-12-14T18:27:23.302 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:23.302 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:23.317 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:23.317 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:23.317 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 12 -lt 100 ']' 2023-12-14T18:27:23.317 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 12 + 1 2023-12-14T18:27:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:22 smithi097 ceph-mon[43486]: pgmap v30: 129 pgs: 129 active+clean; 1.3 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 5.9 KiB/s wr, 7 op/s 2023-12-14T18:27:23.345 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=13 2023-12-14T18:27:23.345 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:23.398 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:23.398 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:23.400 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:23.400 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:23.400 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:24.056 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:24.056 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:24.294 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:24.294 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:24.325 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:24.325 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:24.325 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 13 -lt 100 ']' 2023-12-14T18:27:24.326 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 13 + 1 2023-12-14T18:27:24.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=14 2023-12-14T18:27:24.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:24.392 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:24.392 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:24.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:24.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:24.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:25.014 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:25.014 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.132+0000 7f086a7d4700 1 -- 172.21.15.93:0/959633919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 msgr2=0x7f086410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.132+0000 7f086a7d4700 1 --2- 172.21.15.93:0/959633919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 0x7f086410bfd0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f085c00b0d0 tx=0x7f085c00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.132+0000 7f086a7d4700 1 -- 172.21.15.93:0/959633919 shutdown_connections 2023-12-14T18:27:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.132+0000 7f086a7d4700 1 --2- 172.21.15.93:0/959633919 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 0x7f086410bfd0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.132+0000 7f086a7d4700 1 --2- 172.21.15.93:0/959633919 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08640ff3c0 0x7f08640ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.132+0000 7f086a7d4700 1 --2- 172.21.15.93:0/959633919 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08640ffd60 0x7f0864103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.132+0000 7f086a7d4700 1 -- 172.21.15.93:0/959633919 >> 172.21.15.93:0/959633919 conn(0x7f08640763a0 msgr2=0x7f08640767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:25.136 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.132+0000 7f086a7d4700 1 -- 172.21.15.93:0/959633919 shutdown_connections 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.132+0000 7f086a7d4700 1 -- 172.21.15.93:0/959633919 wait complete. 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f086a7d4700 1 Processor -- start 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f086a7d4700 1 -- start start 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f086a7d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08640ff3c0 0x7f08641972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f086a7d4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08640ffd60 0x7f0864197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f086a7d4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 0x7f08641a2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f086a7d4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0864198120 con 0x7f0864104400 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f086a7d4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641a2890 con 0x7f08640ffd60 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f086a7d4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08641a2a00 con 0x7f08640ff3c0 2023-12-14T18:27:25.137 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f0868d71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 0x7f08641a2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f0868d71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 0x7f08641a2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47390/0 (socket says 172.21.15.93:47390) 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f0868d71700 1 -- 172.21.15.93:0/4245151925 learned_addr learned my addr 172.21.15.93:0/4245151925 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f08637fe700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08640ffd60 0x7f0864197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.133+0000 7f0863fff700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08640ff3c0 0x7f08641972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f0868d71700 1 -- 172.21.15.93:0/4245151925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08640ff3c0 msgr2=0x7f08641972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f0868d71700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08640ff3c0 0x7f08641972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f0868d71700 1 -- 172.21.15.93:0/4245151925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08640ffd60 msgr2=0x7f0864197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f0868d71700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08640ffd60 0x7f0864197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f0868d71700 1 -- 172.21.15.93:0/4245151925 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f085c009d00 con 0x7f0864104400 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f08637fe700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08640ffd60 0x7f0864197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:25.138 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f0868d71700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 0x7f08641a2350 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f085c007870 tx=0x7f085c007950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f08617fa700 1 -- 172.21.15.93:0/4245151925 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c00e040 con 0x7f0864104400 2023-12-14T18:27:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f086a7d4700 1 -- 172.21.15.93:0/4245151925 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f086410f1d0 con 0x7f0864104400 2023-12-14T18:27:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f08617fa700 1 -- 172.21.15.93:0/4245151925 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f085c005070 con 0x7f0864104400 2023-12-14T18:27:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f08617fa700 1 -- 172.21.15.93:0/4245151925 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c01b6c0 con 0x7f0864104400 2023-12-14T18:27:25.139 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.134+0000 7f086a7d4700 1 -- 172.21.15.93:0/4245151925 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086410f040 con 0x7f0864104400 2023-12-14T18:27:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.136+0000 7f08617fa700 1 -- 172.21.15.93:0/4245151925 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f085c0051e0 con 0x7f0864104400 2023-12-14T18:27:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.137+0000 7f08617fa700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f084c0a13e0 0x7f084c0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.137+0000 7f0863fff700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f084c0a13e0 0x7f084c0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.137+0000 7f08617fa700 1 -- 172.21.15.93:0/4245151925 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f085c0ce450 con 0x7f0864104400 2023-12-14T18:27:25.140 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.137+0000 7f0863fff700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f084c0a13e0 0x7f084c0a3890 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0854009bf0 tx=0x7f0854009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.192+0000 7f0876962b80 1 Processor -- start 2023-12-14T18:27:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.192+0000 7f0876962b80 1 -- start start 2023-12-14T18:27:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.192+0000 7f0876962b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 0x561bae77be30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.192+0000 7f0876962b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 0x561bae77c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.202 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.192+0000 7f0876962b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae7870f0 0x561bae7894d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.192+0000 7f0876962b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae666580 con 0x561bae7870f0 2023-12-14T18:27:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.192+0000 7f0876962b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae6d5810 con 0x561bae77c370 2023-12-14T18:27:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.192+0000 7f0876962b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae6ceca0 con 0x561bae5d9430 2023-12-14T18:27:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0868d71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae7870f0 0x561bae7894d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0868d71700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae7870f0 0x561bae7894d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47406/0 (socket says 172.21.15.93:47406) 2023-12-14T18:27:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0868d71700 1 -- 172.21.15.93:0/2782130880 learned_addr learned my addr 172.21.15.93:0/2782130880 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0863fff700 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 0x561bae77be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f08637fe700 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 0x561bae77c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.203 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0868d71700 1 -- 172.21.15.93:0/2782130880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 msgr2=0x561bae77be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0868d71700 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 0x561bae77be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0868d71700 1 -- 172.21.15.93:0/2782130880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 msgr2=0x561bae77c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0868d71700 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 0x561bae77c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0868d71700 1 -- 172.21.15.93:0/2782130880 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bae550350 con 0x561bae7870f0 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f08637fe700 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 0x561bae77c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f0868d71700 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae7870f0 0x561bae7894d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f085c097510 tx=0x7f085c09ab50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5adc4a04fdc73fa server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f084a7fc700 1 -- 172.21.15.93:0/2782130880 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c09f040 con 0x561bae7870f0 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f084a7fc700 1 -- 172.21.15.93:0/2782130880 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f085c0a33d0 con 0x561bae7870f0 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.193+0000 7f084a7fc700 1 -- 172.21.15.93:0/2782130880 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f085c09a500 con 0x561bae7870f0 2023-12-14T18:27:25.204 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.194+0000 7f0876962b80 1 -- 172.21.15.93:0/2782130880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae7870f0 msgr2=0x561bae7894d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.194+0000 7f0876962b80 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae7870f0 0x561bae7894d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f085c097510 tx=0x7f085c09ab50 comp rx=0 tx=0).stop 2023-12-14T18:27:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.194+0000 7f0876962b80 1 -- 172.21.15.93:0/2782130880 shutdown_connections 2023-12-14T18:27:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.194+0000 7f0876962b80 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae7870f0 0x561bae7894d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.194+0000 7f0876962b80 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 0x561bae77be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.194+0000 7f0876962b80 1 --2- 172.21.15.93:0/2782130880 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 0x561bae77c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.194+0000 7f0876962b80 1 -- 172.21.15.93:0/2782130880 >> 172.21.15.93:0/2782130880 conn(0x561bae772890 msgr2=0x561bae778200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.194+0000 7f0876962b80 1 -- 172.21.15.93:0/2782130880 shutdown_connections 2023-12-14T18:27:25.205 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.194+0000 7f0876962b80 1 -- 172.21.15.93:0/2782130880 wait complete. 2023-12-14T18:27:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.195+0000 7f0876962b80 1 -- start start 2023-12-14T18:27:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.195+0000 7f0876962b80 1 Processor -- start 2023-12-14T18:27:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.195+0000 7f0876962b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae786a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.195+0000 7f0876962b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae77c370 0x561bae783d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.195+0000 7f0876962b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae7842c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0876962b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae66af30 con 0x561bae5d9430 2023-12-14T18:27:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0876962b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae66abd0 con 0x561bae7870f0 2023-12-14T18:27:25.206 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0876962b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae552640 con 0x561bae77c370 2023-12-14T18:27:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0876962b80 10 client.0 _ll_register_callbacks cb 0x561bae68e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0863fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae786a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0863fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae786a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47422/0 (socket says 172.21.15.93:47422) 2023-12-14T18:27:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0863fff700 1 -- 172.21.15.93:0/891416400 learned_addr learned my addr 172.21.15.93:0/891416400 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f08637fe700 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae77c370 0x561bae783d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0868d71700 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae7842c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0863fff700 1 -- 172.21.15.93:0/891416400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae77c370 msgr2=0x561bae783d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:25.207 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0863fff700 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae77c370 0x561bae783d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0863fff700 1 -- 172.21.15.93:0/891416400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 msgr2=0x561bae7842c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0863fff700 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae7842c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0863fff700 1 -- 172.21.15.93:0/891416400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f086410f040 con 0x561bae5d9430 2023-12-14T18:27:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0868d71700 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae7842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0863fff700 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae786a00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0854012910 tx=0x7f08540047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e2c99bb8bc82e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f08437fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:27:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f08437fe700 1 -- 172.21.15.93:0/891416400 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854011bb0 con 0x561bae5d9430 2023-12-14T18:27:25.208 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f08437fe700 1 -- 172.21.15.93:0/891416400 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f0854018070 con 0x561bae5d9430 2023-12-14T18:27:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.196+0000 7f0876962b80 1 -- 172.21.15.93:0/891416400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561bae76fea0 con 0x561bae5d9430 2023-12-14T18:27:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f08437fe700 1 -- 172.21.15.93:0/891416400 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0854020840 con 0x561bae5d9430 2023-12-14T18:27:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f0876962b80 10 client.15609 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f0876962b80 1 -- 172.21.15.93:0/891416400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561bae695110 con 0x561bae5d9430 2023-12-14T18:27:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f08437fe700 1 -- 172.21.15.93:0/891416400 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561bae76fea0 con 0x561bae5d9430 2023-12-14T18:27:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f08437fe700 1 -- 172.21.15.93:0/891416400 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f0854029c80 con 0x561bae5d9430 2023-12-14T18:27:25.209 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f0876962b80 10 client.15609 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f0876962b80 10 client.15609 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:27:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f0876962b80 20 client.15609 populate_metadata read hostname 'smithi093' 2023-12-14T18:27:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f0876962b80 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561bae7968e0 0x561bae798cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f0876962b80 4 client.15609 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f0876962b80 1 -- 172.21.15.93:0/891416400 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561bae6c7c80 con 0x561bae7968e0 2023-12-14T18:27:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.197+0000 7f08637fe700 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561bae7968e0 0x561bae798cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.210 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.198+0000 7f08637fe700 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561bae7968e0 0x561bae798cf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9530aff68a206554 server_cookie=4094325b809ca9ca in_seq=0 out_seq=0 2023-12-14T18:27:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.198+0000 7f08437fe700 10 client.15609 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.198+0000 7f08437fe700 1 -- 172.21.15.93:0/891416400 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561bae6c7c80 con 0x561bae7968e0 2023-12-14T18:27:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.198+0000 7f08437fe700 10 client.15609 handle_command_reply: tid=0 2023-12-14T18:27:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 client.15609 shutdown 2023-12-14T18:27:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 -- 172.21.15.93:0/891416400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 msgr2=0x561bae786a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae786a00 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f0854012910 tx=0x7f08540047c0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 -- 172.21.15.93:0/891416400 shutdown_connections 2023-12-14T18:27:25.211 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561bae7968e0 0x561bae798cf0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae786a00 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae7842c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f08437fe700 0 client.15609 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 --2- 172.21.15.93:0/891416400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae77c370 0x561bae783d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 -- 172.21.15.93:0/891416400 >> 172.21.15.93:0/891416400 conn(0x561bae772890 msgr2=0x561bae777d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 -- 172.21.15.93:0/891416400 shutdown_connections 2023-12-14T18:27:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 1 -- 172.21.15.93:0/891416400 wait complete. 2023-12-14T18:27:25.212 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.199+0000 7f0876962b80 20 client.15609 trim_cache size 0 max 16384 2023-12-14T18:27:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f0876962b80 1 Processor -- start 2023-12-14T18:27:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f0876962b80 1 -- start start 2023-12-14T18:27:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f0876962b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 0x561bae78d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f0876962b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae77c370 0x561bae786530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f0876962b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae780f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f0876962b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae77d9d0 con 0x561bae77c370 2023-12-14T18:27:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f0876962b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae7797d0 con 0x561bae7870f0 2023-12-14T18:27:25.221 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f0876962b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae7894d0 con 0x561bae5d9430 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f08637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae77c370 0x561bae786530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f08637fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae77c370 0x561bae786530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47438/0 (socket says 172.21.15.93:47438) 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.214+0000 7f08637fe700 1 -- 172.21.15.93:0/1188550829 learned_addr learned my addr 172.21.15.93:0/1188550829 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f0863fff700 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 0x561bae78d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f0868d71700 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae780f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f08637fe700 1 -- 172.21.15.93:0/1188550829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 msgr2=0x561bae78d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f08637fe700 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 0x561bae78d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f08637fe700 1 -- 172.21.15.93:0/1188550829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 msgr2=0x561bae780f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f08637fe700 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae780f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f08637fe700 1 -- 172.21.15.93:0/1188550829 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561bae694e60 con 0x561bae77c370 2023-12-14T18:27:25.222 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f0868d71700 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae780f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f08637fe700 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae77c370 0x561bae786530 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f086406a3c0 tx=0x7f085801cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfff452b6032b754 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f084affd700 1 -- 172.21.15.93:0/1188550829 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858017040 con 0x561bae77c370 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f084affd700 1 -- 172.21.15.93:0/1188550829 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0858020020 con 0x561bae77c370 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.215+0000 7f084affd700 1 -- 172.21.15.93:0/1188550829 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0858011440 con 0x561bae77c370 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 -- 172.21.15.93:0/1188550829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae77c370 msgr2=0x561bae786530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae77c370 0x561bae786530 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f086406a3c0 tx=0x7f085801cd70 comp rx=0 tx=0).stop 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 -- 172.21.15.93:0/1188550829 shutdown_connections 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae7870f0 0x561bae780f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae5d9430 0x561bae78d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.223 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 --2- 172.21.15.93:0/1188550829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae77c370 0x561bae786530 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 -- 172.21.15.93:0/1188550829 >> 172.21.15.93:0/1188550829 conn(0x561bae772890 msgr2=0x561bae777d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 -- 172.21.15.93:0/1188550829 shutdown_connections 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 -- 172.21.15.93:0/1188550829 wait complete. 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.216+0000 7f0876962b80 1 -- start start 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0876962b80 1 Processor -- start 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0876962b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae7e3f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0876962b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 0x561bae7e4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0876962b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae7870f0 0x561bae7de730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0876962b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae7dedd0 con 0x561bae5d9430 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0876962b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae7def40 con 0x561bae77c370 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0876962b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561bae7df100 con 0x561bae7870f0 2023-12-14T18:27:25.224 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0876962b80 10 client.0 _ll_register_callbacks cb 0x7f08540124e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0863fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae7e3f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0863fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae7e3f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47450/0 (socket says 172.21.15.93:47450) 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0863fff700 1 -- 172.21.15.93:0/3207632153 learned_addr learned my addr 172.21.15.93:0/3207632153 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f0868d71700 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae7870f0 0x561bae7de730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.217+0000 7f08637fe700 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 0x561bae7e4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f0863fff700 1 -- 172.21.15.93:0/3207632153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae7870f0 msgr2=0x561bae7de730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f0863fff700 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae7870f0 0x561bae7de730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f0863fff700 1 -- 172.21.15.93:0/3207632153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 msgr2=0x561bae7e4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f0863fff700 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 0x561bae7e4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.225 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f0863fff700 1 -- 172.21.15.93:0/3207632153 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0858004560 con 0x561bae5d9430 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f0863fff700 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae7e3f50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f085401f430 tx=0x7f085401fc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbb32df1f251f245 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f08437fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f08437fe700 1 -- 172.21.15.93:0/3207632153 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0854004220 con 0x561bae5d9430 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f08437fe700 1 -- 172.21.15.93:0/3207632153 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f0854018040 con 0x561bae5d9430 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f0876962b80 1 -- 172.21.15.93:0/3207632153 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f085c0a33d0 con 0x561bae5d9430 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f08437fe700 1 -- 172.21.15.93:0/3207632153 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f085401fe60 con 0x561bae5d9430 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f0876962b80 10 client.15615 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f0876962b80 1 -- 172.21.15.93:0/3207632153 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x561bae67b940 con 0x561bae5d9430 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f08437fe700 1 -- 172.21.15.93:0/3207632153 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f085c0a33d0 con 0x561bae5d9430 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.218+0000 7f08437fe700 1 -- 172.21.15.93:0/3207632153 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f085403c070 con 0x561bae5d9430 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.219+0000 7f0876962b80 10 client.15615 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:25.226 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.219+0000 7f0876962b80 10 client.15615 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:27:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.219+0000 7f0876962b80 20 client.15615 populate_metadata read hostname 'smithi093' 2023-12-14T18:27:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.219+0000 7f0876962b80 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561bae7ebd20 0x561bae7ee130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.219+0000 7f0876962b80 4 client.15615 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:27:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.219+0000 7f0876962b80 1 -- 172.21.15.93:0/3207632153 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561bae6c79a0 con 0x561bae7ebd20 2023-12-14T18:27:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.219+0000 7f08637fe700 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561bae7ebd20 0x561bae7ee130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.219+0000 7f08437fe700 10 client.15615 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:25.227 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.219+0000 7f08637fe700 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561bae7ebd20 0x561bae7ee130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e8830d7445e2ed4 server_cookie=5117d41a226df74b in_seq=0 out_seq=0 2023-12-14T18:27:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f08437fe700 1 -- 172.21.15.93:0/3207632153 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561bae6c79a0 con 0x561bae7ebd20 2023-12-14T18:27:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f08437fe700 10 client.15615 handle_command_reply: tid=0 2023-12-14T18:27:25.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 client.15615 shutdown 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 -- 172.21.15.93:0/3207632153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 msgr2=0x561bae7e3f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae7e3f50 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f085401f430 tx=0x7f085401fc70 comp rx=0 tx=0).stop 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 -- 172.21.15.93:0/3207632153 shutdown_connections 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561bae7ebd20 0x561bae7ee130 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561bae5d9430 0x561bae7e3f50 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561bae7870f0 0x561bae7de730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 --2- 172.21.15.93:0/3207632153 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561bae77c370 0x561bae7e4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f08437fe700 0 client.15615 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 -- 172.21.15.93:0/3207632153 >> 172.21.15.93:0/3207632153 conn(0x561bae772890 msgr2=0x561bae77de90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:25.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 -- 172.21.15.93:0/3207632153 shutdown_connections 2023-12-14T18:27:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 1 -- 172.21.15.93:0/3207632153 wait complete. 2023-12-14T18:27:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.243+0000 7f0876962b80 20 client.15615 trim_cache size 0 max 16384 2023-12-14T18:27:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:27:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:27:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:27:25.246 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:27:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 -- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f084c0a13e0 msgr2=0x7f084c0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f084c0a13e0 0x7f084c0a3890 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f0854009bf0 tx=0x7f0854009260 comp rx=0 tx=0).stop 2023-12-14T18:27:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 -- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 msgr2=0x7f08641a2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 0x7f08641a2350 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f085c007870 tx=0x7f085c007950 comp rx=0 tx=0).stop 2023-12-14T18:27:25.247 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 -- 172.21.15.93:0/4245151925 shutdown_connections 2023-12-14T18:27:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f084c0a13e0 0x7f084c0a3890 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0864104400 0x7f08641a2350 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f08640ff3c0 0x7f08641972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 --2- 172.21.15.93:0/4245151925 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f08640ffd60 0x7f0864197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 -- 172.21.15.93:0/4245151925 >> 172.21.15.93:0/4245151925 conn(0x7f08640763a0 msgr2=0x7f0864102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 -- 172.21.15.93:0/4245151925 shutdown_connections 2023-12-14T18:27:25.248 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:25.244+0000 7f086a7d4700 1 -- 172.21.15.93:0/4245151925 wait complete. 2023-12-14T18:27:25.248 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:24 smithi160 ceph-mon[43840]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-12-14T18:27:25.302 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:25.302 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:25.314 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:25.314 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:25.314 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 14 -lt 100 ']' 2023-12-14T18:27:25.314 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 14 + 1 2023-12-14T18:27:25.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:24 smithi093 ceph-mon[42070]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-12-14T18:27:25.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:24 smithi097 ceph-mon[43486]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 10 op/s 2023-12-14T18:27:25.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=15 2023-12-14T18:27:25.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:25.417 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:25.417 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:25.418 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:25.418 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:25.419 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:26.089 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:26.089 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:26.285 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:26.285 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:26.297 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:26.297 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:26.297 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 15 -lt 100 ']' 2023-12-14T18:27:26.298 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 15 + 1 2023-12-14T18:27:26.328 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=16 2023-12-14T18:27:26.328 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:26.351 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:26.352 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:26.352 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:26.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:26.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:26.772 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:26.772 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:26.910 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:26.910 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:26.923 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:26.924 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:26.924 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 16 -lt 100 ']' 2023-12-14T18:27:26.924 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 16 + 1 2023-12-14T18:27:26.953 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=17 2023-12-14T18:27:26.953 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:26.976 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:26.976 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:26.978 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:26.978 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:26.978 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:26 smithi160 ceph-mon[43840]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-12-14T18:27:27.260 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:26 smithi093 ceph-mon[42070]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-12-14T18:27:27.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:26 smithi097 ceph-mon[43486]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 11 KiB/s wr, 8 op/s 2023-12-14T18:27:27.380 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:27.380 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:27.551 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:27.552 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:27.590 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:27.590 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:27.590 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 17 -lt 100 ']' 2023-12-14T18:27:27.590 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 17 + 1 2023-12-14T18:27:27.615 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=18 2023-12-14T18:27:27.615 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:27.640 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:27.640 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:27.641 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:27.641 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:27.641 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:28.030 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:28.030 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:28.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:28.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:28.208 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:28.208 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:28.208 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 18 -lt 100 ']' 2023-12-14T18:27:28.208 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 18 + 1 2023-12-14T18:27:28.236 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=19 2023-12-14T18:27:28.236 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:28.265 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:28.265 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:28.266 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:28.266 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:28.266 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:28.788 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:28.789 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:28.968 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:28.968 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:28.981 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:28.981 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:28.981 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 19 -lt 100 ']' 2023-12-14T18:27:28.981 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 19 + 1 2023-12-14T18:27:29.011 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=20 2023-12-14T18:27:29.011 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:29.047 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:29.047 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:29.048 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:29.049 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:29.049 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:29.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:28 smithi093 ceph-mon[42070]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2023-12-14T18:27:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:28 smithi160 ceph-mon[43840]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2023-12-14T18:27:29.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:28 smithi097 ceph-mon[43486]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 82 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 10 KiB/s wr, 6 op/s 2023-12-14T18:27:29.471 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:29.472 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:29.693 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:29.693 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:29.739 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:29.739 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:29.739 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 20 -lt 100 ']' 2023-12-14T18:27:29.739 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 20 + 1 2023-12-14T18:27:29.769 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=21 2023-12-14T18:27:29.769 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:29.797 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:29.797 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:29.799 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:29.799 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:29.799 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:30.263 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:30.263 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:30.443 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:30.443 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:30.455 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:30.456 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:30.456 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 21 -lt 100 ']' 2023-12-14T18:27:30.456 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 21 + 1 2023-12-14T18:27:30.485 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=22 2023-12-14T18:27:30.486 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:30.514 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:30.514 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:30.515 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:30.515 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:30.515 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:31.236 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:30 smithi093 ceph-mon[42070]: pgmap v34: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2023-12-14T18:27:31.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:30 smithi160 ceph-mon[43840]: pgmap v34: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2023-12-14T18:27:31.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:30 smithi097 ceph-mon[43486]: pgmap v34: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 16 op/s 2023-12-14T18:27:31.363 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:31.363 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:31.559 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:31.559 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:31.582 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:31.582 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:31.582 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 22 -lt 100 ']' 2023-12-14T18:27:31.582 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 22 + 1 2023-12-14T18:27:31.622 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=23 2023-12-14T18:27:31.622 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:31.655 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:31.655 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:31.656 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:31.656 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:31.656 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:32.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:31 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:32.260 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:31 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:32.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:31 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:27:32.397 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:32.397 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:32.567 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:32.567 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:32.597 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:32.597 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:32.597 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 23 -lt 100 ']' 2023-12-14T18:27:32.597 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 23 + 1 2023-12-14T18:27:32.643 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=24 2023-12-14T18:27:32.643 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:32.667 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:32.667 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:32.668 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:32.668 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:32.668 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:33.204 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:33.204 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:33.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:32 smithi160 ceph-mon[43840]: pgmap v35: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-12-14T18:27:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:32 smithi093 ceph-mon[42070]: pgmap v35: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-12-14T18:27:33.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:32 smithi097 ceph-mon[43486]: pgmap v35: 129 pgs: 129 active+clean; 1.5 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 13 op/s 2023-12-14T18:27:33.434 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:33.434 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:33.447 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:33.447 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:33.447 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 24 -lt 100 ']' 2023-12-14T18:27:33.447 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 24 + 1 2023-12-14T18:27:33.485 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=25 2023-12-14T18:27:33.485 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:33.565 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:33.565 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:33.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:33.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:33.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:34.329 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:34.329 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:34.659 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:34.659 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:34.706 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:34.706 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:34.706 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 25 -lt 100 ']' 2023-12-14T18:27:34.706 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 25 + 1 2023-12-14T18:27:34.754 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=26 2023-12-14T18:27:34.754 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:34.838 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:34.838 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:34.840 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:34.840 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:34.840 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:35.203 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:34 smithi093 ceph-mon[42070]: pgmap v36: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:27:35.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:34 smithi160 ceph-mon[43840]: pgmap v36: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:27:35.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:34 smithi097 ceph-mon[43486]: pgmap v36: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 25 KiB/s wr, 17 op/s 2023-12-14T18:27:35.346 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:35.346 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:35.492 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:35.492 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:35.508 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:35.508 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:35.508 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 26 -lt 100 ']' 2023-12-14T18:27:35.508 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 26 + 1 2023-12-14T18:27:35.568 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=27 2023-12-14T18:27:35.568 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:35.616 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:35.616 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:35.617 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:35.617 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:35.618 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:36.137 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:36.137 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:36.342 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:36.342 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:36.363 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:36.363 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:36.363 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 27 -lt 100 ']' 2023-12-14T18:27:36.364 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 27 + 1 2023-12-14T18:27:36.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=28 2023-12-14T18:27:36.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:36.406 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:36.406 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:36.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:36.408 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:36.408 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:36 smithi093 ceph-mon[42070]: pgmap v37: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-12-14T18:27:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:36 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:36 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:36 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:27:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:36 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:27:37.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:36 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:37.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:36 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:27:37.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:36 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:27:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:36 smithi097 ceph-mon[43486]: pgmap v37: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-12-14T18:27:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:36 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:36 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:36 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:27:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:36 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:27:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:36 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:36 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:27:37.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:36 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:27:37.220 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:37.220 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:36 smithi160 ceph-mon[43840]: pgmap v37: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 14 op/s 2023-12-14T18:27:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:36 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:27:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:36 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:27:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:36 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:27:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:36 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:27:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:36 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:27:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:36 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:27:37.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:36 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:27:37.383 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:37.383 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:37.397 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:37.397 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:37.397 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 28 -lt 100 ']' 2023-12-14T18:27:37.397 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 28 + 1 2023-12-14T18:27:37.421 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=29 2023-12-14T18:27:37.421 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:37.449 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:37.449 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:37.450 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:37.450 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:37.450 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:37.870 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:37.870 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:37.992 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:37.992 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:38.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:38.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:38.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 29 -lt 100 ']' 2023-12-14T18:27:38.005 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 29 + 1 2023-12-14T18:27:38.021 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=30 2023-12-14T18:27:38.021 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:38.057 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:38.057 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:38.058 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:38.058 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:38.058 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:38.487 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:38.487 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:38.742 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:38.742 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:38.788 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:38.788 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:38.788 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 30 -lt 100 ']' 2023-12-14T18:27:38.789 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 30 + 1 2023-12-14T18:27:38.834 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=31 2023-12-14T18:27:38.835 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:38.875 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:38.875 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:38.876 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:38.877 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:38.877 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:39.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:38 smithi093 ceph-mon[42070]: pgmap v38: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-12-14T18:27:39.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:38 smithi097 ceph-mon[43486]: pgmap v38: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-12-14T18:27:39.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:38 smithi160 ceph-mon[43840]: pgmap v38: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 18 KiB/s wr, 13 op/s 2023-12-14T18:27:39.520 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:39.520 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:39.741 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:39.742 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:39.770 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:39.771 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:39.771 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 31 -lt 100 ']' 2023-12-14T18:27:39.771 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 31 + 1 2023-12-14T18:27:39.843 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=32 2023-12-14T18:27:39.843 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:39.875 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:39.875 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:39.876 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:39.876 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:39.876 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:40.428 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:40.429 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.686+0000 7f46a9fad700 1 -- 172.21.15.93:0/1466566545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 msgr2=0x7f46a410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.686+0000 7f46a9fad700 1 --2- 172.21.15.93:0/1466566545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 0x7f46a410bfd0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f469c00b0d0 tx=0x7f469c00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.686+0000 7f46a9fad700 1 -- 172.21.15.93:0/1466566545 shutdown_connections 2023-12-14T18:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.686+0000 7f46a9fad700 1 --2- 172.21.15.93:0/1466566545 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 0x7f46a410bfd0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.686+0000 7f46a9fad700 1 --2- 172.21.15.93:0/1466566545 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46a40ff3c0 0x7f46a40ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.686+0000 7f46a9fad700 1 --2- 172.21.15.93:0/1466566545 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46a40ffd60 0x7f46a4103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.686+0000 7f46a9fad700 1 -- 172.21.15.93:0/1466566545 >> 172.21.15.93:0/1466566545 conn(0x7f46a40763a0 msgr2=0x7f46a40767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.686+0000 7f46a9fad700 1 -- 172.21.15.93:0/1466566545 shutdown_connections 2023-12-14T18:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.686+0000 7f46a9fad700 1 -- 172.21.15.93:0/1466566545 wait complete. 2023-12-14T18:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.687+0000 7f46a9fad700 1 Processor -- start 2023-12-14T18:27:40.690 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.687+0000 7f46a9fad700 1 -- start start 2023-12-14T18:27:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a9fad700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46a40ff3c0 0x7f46a4196580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a9fad700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46a40ffd60 0x7f46a4196ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a9fad700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 0x7f46a419af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a9fad700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a419b560 con 0x7f46a4104400 2023-12-14T18:27:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a9fad700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a419b6d0 con 0x7f46a40ffd60 2023-12-14T18:27:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a9fad700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a419b890 con 0x7f46a40ff3c0 2023-12-14T18:27:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 0x7f46a419af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 0x7f46a419af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40826/0 (socket says 172.21.15.93:40826) 2023-12-14T18:27:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a3fff700 1 -- 172.21.15.93:0/4147626504 learned_addr learned my addr 172.21.15.93:0/4147626504 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:40.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f469affd700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46a40ffd60 0x7f46a4196ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a37fe700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46a40ff3c0 0x7f46a4196580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a3fff700 1 -- 172.21.15.93:0/4147626504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46a40ff3c0 msgr2=0x7f46a4196580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a3fff700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46a40ff3c0 0x7f46a4196580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a3fff700 1 -- 172.21.15.93:0/4147626504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46a40ffd60 msgr2=0x7f46a4196ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a3fff700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46a40ffd60 0x7f46a4196ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a3fff700 1 -- 172.21.15.93:0/4147626504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f469c009d00 con 0x7f46a4104400 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a3fff700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 0x7f46a419af20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f469c007890 tx=0x7f469c0078c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a17fa700 1 -- 172.21.15.93:0/4147626504 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c00e040 con 0x7f46a4104400 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.688+0000 7f46a17fa700 1 -- 172.21.15.93:0/4147626504 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469c00bd30 con 0x7f46a4104400 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.689+0000 7f46a9fad700 1 -- 172.21.15.93:0/4147626504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a410f220 con 0x7f46a4104400 2023-12-14T18:27:40.694 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.689+0000 7f46a17fa700 1 -- 172.21.15.93:0/4147626504 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c01d470 con 0x7f46a4104400 2023-12-14T18:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.689+0000 7f46a9fad700 1 -- 172.21.15.93:0/4147626504 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a410f090 con 0x7f46a4104400 2023-12-14T18:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.691+0000 7f46a17fa700 1 -- 172.21.15.93:0/4147626504 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135948+0+0 (secure 0 0 0) 0x7f469c004a50 con 0x7f46a4104400 2023-12-14T18:27:40.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.691+0000 7f46a17fa700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f46900a13e0 0x7f46900a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.691+0000 7f46a37fe700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f46900a13e0 0x7f46900a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.691+0000 7f46a17fa700 1 -- 172.21.15.93:0/4147626504 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f469c0ce3e0 con 0x7f46a4104400 2023-12-14T18:27:40.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.692+0000 7f46a37fe700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f46900a13e0 0x7f46900a3890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4694009c60 tx=0x7f4694009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.749 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:40.750 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.785+0000 7f46b613bb80 1 Processor -- start 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.785+0000 7f46b613bb80 1 -- start start 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c6fce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 0x55c92c6fd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c7080f0 0x55c92c70a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46b613bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c5e7580 con 0x55c92c7080f0 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46b613bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c656810 con 0x55c92c55a430 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46b613bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c64fca0 con 0x55c92c6fd370 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c7080f0 0x55c92c70a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c7080f0 0x55c92c70a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40832/0 (socket says 172.21.15.93:40832) 2023-12-14T18:27:40.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a3fff700 1 -- 172.21.15.93:0/2657116747 learned_addr learned my addr 172.21.15.93:0/2657116747 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f469affd700 1 --2- 172.21.15.93:0/2657116747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 0x55c92c6fd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a37fe700 1 --2- 172.21.15.93:0/2657116747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c6fce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a3fff700 1 -- 172.21.15.93:0/2657116747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 msgr2=0x55c92c6fd7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a3fff700 1 --2- 172.21.15.93:0/2657116747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 0x55c92c6fd7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a3fff700 1 -- 172.21.15.93:0/2657116747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 msgr2=0x55c92c6fce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a3fff700 1 --2- 172.21.15.93:0/2657116747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c6fce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a3fff700 1 -- 172.21.15.93:0/2657116747 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92c4d1350 con 0x55c92c7080f0 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.786+0000 7f46a3fff700 1 --2- 172.21.15.93:0/2657116747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c7080f0 0x55c92c70a4d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f469c007930 tx=0x7f469c005420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b5edcffd4853c81 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.787+0000 7f4699ffb700 1 -- 172.21.15.93:0/2657116747 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c09b000 con 0x55c92c7080f0 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.787+0000 7f4699ffb700 1 -- 172.21.15.93:0/2657116747 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469c09f030 con 0x55c92c7080f0 2023-12-14T18:27:40.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.787+0000 7f4699ffb700 1 -- 172.21.15.93:0/2657116747 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f469c024e30 con 0x55c92c7080f0 2023-12-14T18:27:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.787+0000 7f46b613bb80 1 -- 172.21.15.93:0/2657116747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c7080f0 msgr2=0x55c92c70a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.787+0000 7f46b613bb80 1 --2- 172.21.15.93:0/2657116747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c7080f0 0x55c92c70a4d0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f469c007930 tx=0x7f469c005420 comp rx=0 tx=0).stop 2023-12-14T18:27:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.788+0000 7f46b613bb80 1 -- 172.21.15.93:0/2657116747 shutdown_connections 2023-12-14T18:27:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.788+0000 7f46b613bb80 1 --2- 172.21.15.93:0/2657116747 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c7080f0 0x55c92c70a4d0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.788+0000 7f46b613bb80 1 --2- 172.21.15.93:0/2657116747 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c6fce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.788+0000 7f46b613bb80 1 --2- 172.21.15.93:0/2657116747 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 0x55c92c6fd7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.788+0000 7f46b613bb80 1 -- 172.21.15.93:0/2657116747 >> 172.21.15.93:0/2657116747 conn(0x55c92c6f3890 msgr2=0x55c92c6f9200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.788+0000 7f46b613bb80 1 -- 172.21.15.93:0/2657116747 shutdown_connections 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.788+0000 7f46b613bb80 1 -- 172.21.15.93:0/2657116747 wait complete. 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.788+0000 7f46b613bb80 1 -- start start 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.788+0000 7f46b613bb80 1 Processor -- start 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c707a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 0x55c92c7053b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46b613bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c5ebf30 con 0x55c92c6fd370 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46b613bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c5ebbd0 con 0x55c92c55a430 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46b613bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c4d3640 con 0x55c92c7080f0 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46b613bb80 10 client.0 _ll_register_callbacks cb 0x55c92c60f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f469affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f469affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40840/0 (socket says 172.21.15.93:40840) 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f469affd700 1 -- 172.21.15.93:0/414725602 learned_addr learned my addr 172.21.15.93:0/414725602 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:40.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46a37fe700 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c707a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46a3fff700 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 0x55c92c7053b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f469affd700 1 -- 172.21.15.93:0/414725602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 msgr2=0x55c92c7053b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f469affd700 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 0x55c92c7053b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f469affd700 1 -- 172.21.15.93:0/414725602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 msgr2=0x55c92c707a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f469affd700 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c707a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f469affd700 1 -- 172.21.15.93:0/414725602 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a410f090 con 0x55c92c6fd370 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46a3fff700 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 0x55c92c7053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f46a37fe700 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c707a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f469affd700 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f468c00be70 tx=0x7f468c011720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26bbefac5df8f6bf server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.789+0000 7f4682ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f4682ffd700 1 -- 172.21.15.93:0/414725602 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c016070 con 0x55c92c6fd370 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f4682ffd700 1 -- 172.21.15.93:0/414725602 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f468c004830 con 0x55c92c6fd370 2023-12-14T18:27:40.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f46b613bb80 1 -- 172.21.15.93:0/414725602 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c92c6f0ea0 con 0x55c92c6fd370 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f4682ffd700 1 -- 172.21.15.93:0/414725602 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f468c0049f0 con 0x55c92c6fd370 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f46b613bb80 10 client.15627 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f46b613bb80 1 -- 172.21.15.93:0/414725602 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c92c616110 con 0x55c92c6fd370 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f4682ffd700 1 -- 172.21.15.93:0/414725602 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c92c6f0ea0 con 0x55c92c6fd370 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f4682ffd700 1 -- 172.21.15.93:0/414725602 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f468c01a850 con 0x55c92c6fd370 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f46b613bb80 10 client.15627 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f46b613bb80 10 client.15627 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f46b613bb80 20 client.15627 populate_metadata read hostname 'smithi093' 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f46b613bb80 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c92c717910 0x55c92c719d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f46b613bb80 4 client.15627 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.790+0000 7f46b613bb80 1 -- 172.21.15.93:0/414725602 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c92c648c80 con 0x55c92c717910 2023-12-14T18:27:40.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.791+0000 7f46a37fe700 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c92c717910 0x55c92c719d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.791+0000 7f46a37fe700 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c92c717910 0x55c92c719d20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9b8fa5b6f954f702 server_cookie=5cffd8614dafd6ad in_seq=0 out_seq=0 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.791+0000 7f4682ffd700 10 client.15627 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f4682ffd700 1 -- 172.21.15.93:0/414725602 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c92c648c80 con 0x55c92c717910 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f4682ffd700 10 client.15627 handle_command_reply: tid=0 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 client.15627 shutdown 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 -- 172.21.15.93:0/414725602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 msgr2=0x55c92c704d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704d80 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f468c00be70 tx=0x7f468c011720 comp rx=0 tx=0).stop 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 -- 172.21.15.93:0/414725602 shutdown_connections 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c92c717910 0x55c92c719d20 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 0x55c92c7053b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c707a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 --2- 172.21.15.93:0/414725602 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704d80 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f4682ffd700 0 client.15627 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 -- 172.21.15.93:0/414725602 >> 172.21.15.93:0/414725602 conn(0x55c92c6f3890 msgr2=0x55c92c6f8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 -- 172.21.15.93:0/414725602 shutdown_connections 2023-12-14T18:27:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 1 -- 172.21.15.93:0/414725602 wait complete. 2023-12-14T18:27:40.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.792+0000 7f46b613bb80 20 client.15627 trim_cache size 0 max 16384 2023-12-14T18:27:40.800 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:40.800 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:40.800 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 32 -lt 100 ']' 2023-12-14T18:27:40.800 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 32 + 1 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f46b613bb80 1 Processor -- start 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f46b613bb80 1 -- start start 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c707590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 0x55c92c704a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f46b613bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c6f3290 con 0x55c92c6fd370 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f46b613bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c6fddc0 con 0x55c92c55a430 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f46b613bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c6fe0a0 con 0x55c92c7080f0 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f469affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f469affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40852/0 (socket says 172.21.15.93:40852) 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.808+0000 7f469affd700 1 -- 172.21.15.93:0/1262950562 learned_addr learned my addr 172.21.15.93:0/1262950562 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f46a37fe700 1 --2- 172.21.15.93:0/1262950562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c707590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f46a3fff700 1 --2- 172.21.15.93:0/1262950562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 0x55c92c704a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f469affd700 1 -- 172.21.15.93:0/1262950562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 msgr2=0x55c92c704a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f469affd700 1 --2- 172.21.15.93:0/1262950562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 0x55c92c704a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f469affd700 1 -- 172.21.15.93:0/1262950562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 msgr2=0x55c92c707590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f469affd700 1 --2- 172.21.15.93:0/1262950562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c707590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f469affd700 1 -- 172.21.15.93:0/1262950562 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c92c615e60 con 0x55c92c6fd370 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f469affd700 1 --2- 172.21.15.93:0/1262950562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704540 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f468c01a040 tx=0x7f468c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7f176e81ba8d71d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f469a7fc700 1 -- 172.21.15.93:0/1262950562 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c016070 con 0x55c92c6fd370 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f469a7fc700 1 -- 172.21.15.93:0/1262950562 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f468c01a8d0 con 0x55c92c6fd370 2023-12-14T18:27:40.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.809+0000 7f469a7fc700 1 -- 172.21.15.93:0/1262950562 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f468c010960 con 0x55c92c6fd370 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 -- 172.21.15.93:0/1262950562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 msgr2=0x55c92c704540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 --2- 172.21.15.93:0/1262950562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704540 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f468c01a040 tx=0x7f468c011ba0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 -- 172.21.15.93:0/1262950562 shutdown_connections 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 --2- 172.21.15.93:0/1262950562 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c7080f0 0x55c92c704a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 --2- 172.21.15.93:0/1262950562 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c55a430 0x55c92c707590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 --2- 172.21.15.93:0/1262950562 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c6fd370 0x55c92c704540 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 -- 172.21.15.93:0/1262950562 >> 172.21.15.93:0/1262950562 conn(0x55c92c6f3890 msgr2=0x55c92c713c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 -- 172.21.15.93:0/1262950562 shutdown_connections 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 -- 172.21.15.93:0/1262950562 wait complete. 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.810+0000 7f46b613bb80 1 -- start start 2023-12-14T18:27:40.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46b613bb80 1 Processor -- start 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c55a430 0x55c92c70ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 0x55c92c70cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46b613bb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c7080f0 0x55c92c70d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46b613bb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c70d7d0 con 0x55c92c55a430 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46b613bb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c70d940 con 0x55c92c7080f0 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46b613bb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c92c70db00 con 0x55c92c6fd370 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46b613bb80 10 client.0 _ll_register_callbacks cb 0x7f468c00b940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c55a430 0x55c92c70ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c55a430 0x55c92c70ea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:40856/0 (socket says 172.21.15.93:40856) 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a37fe700 1 -- 172.21.15.93:0/658385912 learned_addr learned my addr 172.21.15.93:0/658385912 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f469affd700 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 0x55c92c70cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a3fff700 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c7080f0 0x55c92c70d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a37fe700 1 -- 172.21.15.93:0/658385912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 msgr2=0x55c92c70cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:40.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a37fe700 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 0x55c92c70cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a37fe700 1 -- 172.21.15.93:0/658385912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c7080f0 msgr2=0x55c92c70d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a37fe700 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c7080f0 0x55c92c70d1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a37fe700 1 -- 172.21.15.93:0/658385912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c92c616110 con 0x55c92c55a430 2023-12-14T18:27:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.811+0000 7f46a3fff700 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c7080f0 0x55c92c70d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46a37fe700 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c55a430 0x55c92c70ea00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4694013010 tx=0x7f46940047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a1dcf16cdacca4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f4682ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:27:40.818 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f4682ffd700 1 -- 172.21.15.93:0/658385912 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4694023040 con 0x55c92c55a430 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f4682ffd700 1 -- 172.21.15.93:0/658385912 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f469401e930 con 0x55c92c55a430 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46b613bb80 1 -- 172.21.15.93:0/658385912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f469c09f030 con 0x55c92c55a430 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f4682ffd700 1 -- 172.21.15.93:0/658385912 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f469401eaf0 con 0x55c92c55a430 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46b613bb80 10 client.15633 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46b613bb80 1 -- 172.21.15.93:0/658385912 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55c92c5fc940 con 0x55c92c55a430 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f4682ffd700 1 -- 172.21.15.93:0/658385912 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f469c09f030 con 0x55c92c55a430 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f4682ffd700 1 -- 172.21.15.93:0/658385912 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f46940333f0 con 0x55c92c55a430 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46b613bb80 10 client.15633 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46b613bb80 10 client.15633 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46b613bb80 20 client.15633 populate_metadata read hostname 'smithi093' 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46b613bb80 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c92c771200 0x55c92c773610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46b613bb80 4 client.15633 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.812+0000 7f46b613bb80 1 -- 172.21.15.93:0/658385912 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c92c6489a0 con 0x55c92c771200 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.813+0000 7f469affd700 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c92c771200 0x55c92c773610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:40.819 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.813+0000 7f4682ffd700 10 client.15633 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:40.820 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.813+0000 7f469affd700 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c92c771200 0x55c92c773610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67ab1d248761cca2 server_cookie=ffa71bf7e9a1d60f in_seq=0 out_seq=0 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.838+0000 7f4682ffd700 1 -- 172.21.15.93:0/658385912 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c92c6489a0 con 0x55c92c771200 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.838+0000 7f4682ffd700 10 client.15633 handle_command_reply: tid=0 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.838+0000 7f46b613bb80 1 client.15633 shutdown 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 -- 172.21.15.93:0/658385912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c55a430 msgr2=0x55c92c70ea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c55a430 0x55c92c70ea00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f4694013010 tx=0x7f46940047c0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 -- 172.21.15.93:0/658385912 shutdown_connections 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c92c771200 0x55c92c773610 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c92c55a430 0x55c92c70ea00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c92c7080f0 0x55c92c70d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f4682ffd700 0 client.15633 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 --2- 172.21.15.93:0/658385912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c92c6fd370 0x55c92c70cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 -- 172.21.15.93:0/658385912 >> 172.21.15.93:0/658385912 conn(0x55c92c6f3890 msgr2=0x55c92c6fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.842 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 -- 172.21.15.93:0/658385912 shutdown_connections 2023-12-14T18:27:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 1 -- 172.21.15.93:0/658385912 wait complete. 2023-12-14T18:27:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46b613bb80 20 client.15633 trim_cache size 0 max 16384 2023-12-14T18:27:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:27:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:27:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:27:40.843 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46a9fad700 1 -- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f46900a13e0 msgr2=0x7f46900a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46a9fad700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f46900a13e0 0x7f46900a3890 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f4694009c60 tx=0x7f4694009260 comp rx=0 tx=0).stop 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46a9fad700 1 -- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 msgr2=0x7f46a419af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.839+0000 7f46a9fad700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 0x7f46a419af20 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f469c007890 tx=0x7f469c0078c0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.840+0000 7f46a9fad700 1 -- 172.21.15.93:0/4147626504 shutdown_connections 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.840+0000 7f46a9fad700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f46900a13e0 0x7f46900a3890 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.840+0000 7f46a9fad700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f46a4104400 0x7f46a419af20 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.840+0000 7f46a9fad700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f46a40ff3c0 0x7f46a4196580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.840+0000 7f46a9fad700 1 --2- 172.21.15.93:0/4147626504 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f46a40ffd60 0x7f46a4196ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.840+0000 7f46a9fad700 1 -- 172.21.15.93:0/4147626504 >> 172.21.15.93:0/4147626504 conn(0x7f46a40763a0 msgr2=0x7f46a4068060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:40.844 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.840+0000 7f46a9fad700 1 -- 172.21.15.93:0/4147626504 shutdown_connections 2023-12-14T18:27:40.845 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:40.840+0000 7f46a9fad700 1 -- 172.21.15.93:0/4147626504 wait complete. 2023-12-14T18:27:40.892 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=33 2023-12-14T18:27:40.892 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:40.942 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:40.942 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:40.943 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:40.943 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:40.943 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:41.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:40 smithi093 ceph-mon[42070]: pgmap v39: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-12-14T18:27:41.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:40 smithi097 ceph-mon[43486]: pgmap v39: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-12-14T18:27:41.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:40 smithi160 ceph-mon[43840]: pgmap v39: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 24 KiB/s rd, 23 KiB/s wr, 17 op/s 2023-12-14T18:27:41.336 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:41.336 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:42.024 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:42.025 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:42.038 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:42.038 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:42.038 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 33 -lt 100 ']' 2023-12-14T18:27:42.038 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 33 + 1 2023-12-14T18:27:42.109 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=34 2023-12-14T18:27:42.109 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:42.199 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:42.199 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:42.200 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:42.200 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:42.200 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:43.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:42 smithi097 ceph-mon[43486]: pgmap v40: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-14T18:27:43.138 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:42 smithi093 ceph-mon[42070]: pgmap v40: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-14T18:27:43.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:42 smithi160 ceph-mon[43840]: pgmap v40: 129 pgs: 129 active+clean; 1.6 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-14T18:27:43.353 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:43.353 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:44.382 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:44.382 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:44.438 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:44.438 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:44.438 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 34 -lt 100 ']' 2023-12-14T18:27:44.438 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 34 + 1 2023-12-14T18:27:44.529 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=35 2023-12-14T18:27:44.529 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:44.553 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:44.554 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:44.554 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:44.555 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:44.555 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:45.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:44 smithi093 ceph-mon[42070]: pgmap v41: 129 pgs: 129 active+clean; 1.7 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-12-14T18:27:45.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:44 smithi097 ceph-mon[43486]: pgmap v41: 129 pgs: 129 active+clean; 1.7 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-12-14T18:27:45.237 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:45.237 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:45.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:44 smithi160 ceph-mon[43840]: pgmap v41: 129 pgs: 129 active+clean; 1.7 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-12-14T18:27:45.549 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:45.549 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:45.562 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:45.562 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:45.562 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 35 -lt 100 ']' 2023-12-14T18:27:45.563 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 35 + 1 2023-12-14T18:27:45.617 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=36 2023-12-14T18:27:45.617 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:45.644 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:45.644 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:45.646 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:45.646 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:45.646 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:46.969 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:46.969 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:46 smithi093 ceph-mon[42070]: pgmap v42: 129 pgs: 129 active+clean; 1.7 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-12-14T18:27:47.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:46 smithi097 ceph-mon[43486]: pgmap v42: 129 pgs: 129 active+clean; 1.7 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-12-14T18:27:47.190 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:47.190 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:47.205 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:47.205 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:47.205 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 36 -lt 100 ']' 2023-12-14T18:27:47.205 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 36 + 1 2023-12-14T18:27:47.233 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=37 2023-12-14T18:27:47.233 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:46 smithi160 ceph-mon[43840]: pgmap v42: 129 pgs: 129 active+clean; 1.7 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-12-14T18:27:47.262 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:47.262 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:47.263 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:47.263 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:47.264 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:47.735 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:47.735 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:47.882 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:47.882 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:47.894 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:47.895 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:47.895 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 37 -lt 100 ']' 2023-12-14T18:27:47.895 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 37 + 1 2023-12-14T18:27:47.925 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=38 2023-12-14T18:27:47.925 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:47.953 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:47.953 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:47.955 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:47.955 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:47.955 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:48.286 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:48.286 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:48.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:48.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:48.462 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:48.462 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:48.462 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 38 -lt 100 ']' 2023-12-14T18:27:48.462 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 38 + 1 2023-12-14T18:27:48.491 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=39 2023-12-14T18:27:48.491 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:48.520 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:48.520 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:48.521 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:48.521 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:48.521 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:48.877 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:48.878 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:49.023 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:49.023 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:49.036 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:49.036 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:49.036 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 39 -lt 100 ']' 2023-12-14T18:27:49.036 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 39 + 1 2023-12-14T18:27:49.058 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=40 2023-12-14T18:27:49.058 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:49.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:48 smithi093 ceph-mon[42070]: pgmap v43: 129 pgs: 129 active+clean; 1.7 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-12-14T18:27:49.089 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:49.089 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:49.090 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:49.090 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:49.090 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:49.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:48 smithi097 ceph-mon[43486]: pgmap v43: 129 pgs: 129 active+clean; 1.7 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-12-14T18:27:49.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:48 smithi160 ceph-mon[43840]: pgmap v43: 129 pgs: 129 active+clean; 1.7 MiB data, 83 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-12-14T18:27:49.669 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:49.669 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:49.848 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:49.848 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:49.862 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:49.862 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:49.862 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 40 -lt 100 ']' 2023-12-14T18:27:49.862 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 40 + 1 2023-12-14T18:27:49.894 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=41 2023-12-14T18:27:49.895 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:49.912 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:49.912 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:49.913 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:49.913 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:49.913 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:50.352 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:50.352 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:50.498 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:50.498 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:50.511 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:50.511 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:50.511 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 41 -lt 100 ']' 2023-12-14T18:27:50.511 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 41 + 1 2023-12-14T18:27:50.536 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=42 2023-12-14T18:27:50.536 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:50.585 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:50.585 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:50.586 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:50.586 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:50.586 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:51.018 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:51.018 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:51.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:50 smithi097 ceph-mon[43486]: pgmap v44: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-12-14T18:27:51.198 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:51.198 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:51.211 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:51.211 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:51.211 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 42 -lt 100 ']' 2023-12-14T18:27:51.211 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 42 + 1 2023-12-14T18:27:51.241 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=43 2023-12-14T18:27:51.241 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:50 smithi160 ceph-mon[43840]: pgmap v44: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-12-14T18:27:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:50 smithi093 ceph-mon[42070]: pgmap v44: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 22 KiB/s wr, 15 op/s 2023-12-14T18:27:51.763 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:51.764 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:51.765 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:51.765 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:51.765 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:52.193 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:52.193 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:52.340 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:52.340 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:52.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:52.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:52.353 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 43 -lt 100 ']' 2023-12-14T18:27:52.353 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 43 + 1 2023-12-14T18:27:52.383 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=44 2023-12-14T18:27:52.383 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:52.406 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:52.407 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:52.408 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:52.408 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:52.408 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:52.852 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:52.852 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:53.031 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:53.031 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:53.044 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:53.045 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:53.045 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 44 -lt 100 ']' 2023-12-14T18:27:53.045 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 44 + 1 2023-12-14T18:27:53.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:52 smithi093 ceph-mon[42070]: pgmap v45: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-12-14T18:27:53.082 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=45 2023-12-14T18:27:53.082 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:53.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:52 smithi097 ceph-mon[43486]: pgmap v45: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-12-14T18:27:53.113 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:53.113 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:53.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:53.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:53.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:52 smithi160 ceph-mon[43840]: pgmap v45: 129 pgs: 129 active+clean; 1.8 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 17 KiB/s wr, 12 op/s 2023-12-14T18:27:53.468 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:53.468 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:53.614 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:53.614 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:53.628 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:53.628 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:53.628 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 45 -lt 100 ']' 2023-12-14T18:27:53.628 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 45 + 1 2023-12-14T18:27:53.657 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=46 2023-12-14T18:27:53.658 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:53.680 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:53.680 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:53.681 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:53.682 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:53.682 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:54.218 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:54.218 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:54.532 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:54.532 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:54.585 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:54.585 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:54.585 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 46 -lt 100 ']' 2023-12-14T18:27:54.585 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 46 + 1 2023-12-14T18:27:54.632 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=47 2023-12-14T18:27:54.632 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:54.689 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:54.689 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:54.690 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:54.691 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:54.691 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:55.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:54 smithi097 ceph-mon[43486]: pgmap v46: 129 pgs: 129 active+clean; 1.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-12-14T18:27:55.239 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:54 smithi093 ceph-mon[42070]: pgmap v46: 129 pgs: 129 active+clean; 1.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-12-14T18:27:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:54 smithi160 ceph-mon[43840]: pgmap v46: 129 pgs: 129 active+clean; 1.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 24 KiB/s wr, 15 op/s 2023-12-14T18:27:55.392 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:55.393 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:55.698 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:55.698 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:55.720 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:55.720 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:55.720 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 47 -lt 100 ']' 2023-12-14T18:27:55.720 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 47 + 1 2023-12-14T18:27:55.748 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=48 2023-12-14T18:27:55.748 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:55.776 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:55.776 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:55.777 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:55.777 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:55.777 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:56.384 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:56.384 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:56.672 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:56.672 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.693+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2036840125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e41015f0 msgr2=0x7fb3e4104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.693+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2036840125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e41015f0 0x7fb3e4104e80 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80099c0 tx=0x7fb3d8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.693+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2036840125 shutdown_connections 2023-12-14T18:27:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.693+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2036840125 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e4105560 0x7fb3e410d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.693+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2036840125 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e4100c50 0x7fb3e4101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.693+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2036840125 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e41015f0 0x7fb3e4104e80 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.693+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2036840125 >> 172.21.15.93:0/2036840125 conn(0x7fb3e4077590 msgr2=0x7fb3e4077990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.693+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2036840125 shutdown_connections 2023-12-14T18:27:56.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.693+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2036840125 wait complete. 2023-12-14T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.694+0000 7fb3e97c7700 1 Processor -- start 2023-12-14T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.694+0000 7fb3e97c7700 1 -- start start 2023-12-14T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.694+0000 7fb3e97c7700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e4100c50 0x7fb3e419e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.694+0000 7fb3e97c7700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e4105560 0x7fb3e419eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.694+0000 7fb3e97c7700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e4198640 0x7fb3e4198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.694+0000 7fb3e97c7700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41990b0 con 0x7fb3e4100c50 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.694+0000 7fb3e97c7700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41991f0 con 0x7fb3e4105560 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.694+0000 7fb3e97c7700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3e41993b0 con 0x7fb3e4198640 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e4100c50 0x7fb3e419e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e4100c50 0x7fb3e419e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46872/0 (socket says 172.21.15.93:46872) 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e4105560 0x7fb3e419eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/2956579943 learned_addr learned my addr 172.21.15.93:0/2956579943 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e37fe700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e4198640 0x7fb3e4198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/2956579943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e4198640 msgr2=0x7fb3e4198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e4198640 0x7fb3e4198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/2956579943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e4105560 msgr2=0x7fb3e419eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e4105560 0x7fb3e419eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/2956579943 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3d80095f0 con 0x7fb3e4100c50 2023-12-14T18:27:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e4105560 0x7fb3e419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e4100c50 0x7fb3e419e5d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400d980 tx=0x7fb3d400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3cbfff700 1 -- 172.21.15.93:0/2956579943 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4004220 con 0x7fb3e4100c50 2023-12-14T18:27:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2956579943 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3e4110290 con 0x7fb3e4100c50 2023-12-14T18:27:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.695+0000 7fb3cbfff700 1 -- 172.21.15.93:0/2956579943 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3d4007640 con 0x7fb3e4100c50 2023-12-14T18:27:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.696+0000 7fb3cbfff700 1 -- 172.21.15.93:0/2956579943 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d4005690 con 0x7fb3e4100c50 2023-12-14T18:27:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.696+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2956579943 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e4110100 con 0x7fb3e4100c50 2023-12-14T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.698+0000 7fb3cbfff700 1 -- 172.21.15.93:0/2956579943 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 24) v1 ==== 135948+0+0 (secure 0 0 0) 0x7fb3d4004750 con 0x7fb3e4100c50 2023-12-14T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.699+0000 7fb3cbfff700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb3cc0a13e0 0x7fb3cc0a3890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.699+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb3cc0a13e0 0x7fb3cc0a3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.699+0000 7fb3cbfff700 1 -- 172.21.15.93:0/2956579943 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb3d40cd100 con 0x7fb3e4100c50 2023-12-14T18:27:56.701 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:56.702 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:56.702 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 48 -lt 100 ']' 2023-12-14T18:27:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.699+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb3cc0a13e0 0x7fb3cc0a3890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8008000 tx=0x7fb3d8003680 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:56.702 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 48 + 1 2023-12-14T18:27:56.773 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=49 2023-12-14T18:27:56.774 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3f5955b80 1 Processor -- start 2023-12-14T18:27:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3f5955b80 1 -- start start 2023-12-14T18:27:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c0e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c17e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9ce4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7e8ab580 con 0x55eb7e81e430 2023-12-14T18:27:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7e91a810 con 0x55eb7e9cc0f0 2023-12-14T18:27:56.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7e913ca0 con 0x55eb7e9c1370 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c0e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c0e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46876/0 (socket says 172.21.15.93:46876) 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/2210315236 learned_addr learned my addr 172.21.15.93:0/2210315236 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3e37fe700 1 --2- 172.21.15.93:0/2210315236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9ce4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/2210315236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c17e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.801+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/2210315236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 msgr2=0x55eb7e9c17e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.802+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/2210315236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c17e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.802+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/2210315236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9cc0f0 msgr2=0x55eb7e9ce4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.802+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/2210315236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9ce4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.802+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/2210315236 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb7e795350 con 0x55eb7e81e430 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.802+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/2210315236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c0e30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4007e80 tx=0x7fb3d40960a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b42976fc4be2607b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.802+0000 7fb3c97fa700 1 -- 172.21.15.93:0/2210315236 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d409d070 con 0x55eb7e81e430 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.802+0000 7fb3c97fa700 1 -- 172.21.15.93:0/2210315236 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3d409dc60 con 0x55eb7e81e430 2023-12-14T18:27:56.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.802+0000 7fb3c97fa700 1 -- 172.21.15.93:0/2210315236 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d40b2450 con 0x55eb7e81e430 2023-12-14T18:27:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 -- 172.21.15.93:0/2210315236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 msgr2=0x55eb7e9c0e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/2210315236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c0e30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7fb3d4007e80 tx=0x7fb3d40960a0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 -- 172.21.15.93:0/2210315236 shutdown_connections 2023-12-14T18:27:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/2210315236 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c0e30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/2210315236 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9ce4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/2210315236 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c17e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 -- 172.21.15.93:0/2210315236 >> 172.21.15.93:0/2210315236 conn(0x55eb7e9b7890 msgr2=0x55eb7e9bd200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 -- 172.21.15.93:0/2210315236 shutdown_connections 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 -- 172.21.15.93:0/2210315236 wait complete. 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.803+0000 7fb3f5955b80 1 -- start start 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3f5955b80 1 Processor -- start 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 0x55eb7e9cba70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7e8aff30 con 0x55eb7e9cc0f0 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7e8afbd0 con 0x55eb7e81e430 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7e797640 con 0x55eb7e9c1370 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3f5955b80 10 client.0 _ll_register_callbacks cb 0x55eb7e8d38b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 0x55eb7e9cba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46886/0 (socket says 172.21.15.93:46886) 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e37fe700 1 -- 172.21.15.93:0/1109688495 learned_addr learned my addr 172.21.15.93:0/1109688495 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/1109688495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 msgr2=0x55eb7e9c8d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c8d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/1109688495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9cc0f0 msgr2=0x55eb7e9c93c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c93c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/1109688495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3e4110100 con 0x55eb7e81e430 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.804+0000 7fb3e37fe700 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.805+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 0x55eb7e9cba70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40966a0 tx=0x7fb3d4005e30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf311d478631722f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.805+0000 7fb3c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.805+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1109688495 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d409d050 con 0x55eb7e81e430 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.805+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1109688495 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb3d40ca370 con 0x55eb7e81e430 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.805+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1109688495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55eb7e9b4ea0 con 0x55eb7e81e430 2023-12-14T18:27:56.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.805+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1109688495 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3d40b2420 con 0x55eb7e81e430 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.805+0000 7fb3f5955b80 10 client.25283 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.805+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1109688495 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55eb7e8da110 con 0x55eb7e81e430 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.805+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1109688495 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55eb7e9b4ea0 con 0x55eb7e81e430 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1109688495 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb3d40ca660 con 0x55eb7e81e430 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3f5955b80 10 client.25283 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3f5955b80 10 client.25283 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3f5955b80 20 client.25283 populate_metadata read hostname 'smithi093' 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55eb7e9db670 0x55eb7e9dda50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3f5955b80 4 client.25283 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1109688495 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55eb7e90cc80 con 0x55eb7e9db670 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55eb7e9db670 0x55eb7e9dda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55eb7e9db670 0x55eb7e9dda50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cde7db11aa22ab41 server_cookie=ced7bafeaf0fadb9 in_seq=0 out_seq=0 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.806+0000 7fb3c6ff5700 10 client.25283 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:56.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1109688495 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55eb7e90cc80 con 0x55eb7e9db670 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3c6ff5700 10 client.25283 handle_command_reply: tid=0 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 client.25283 shutdown 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1109688495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 msgr2=0x55eb7e9cba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 0x55eb7e9cba70 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb3d40966a0 tx=0x7fb3d4005e30 comp rx=0 tx=0).stop 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1109688495 shutdown_connections 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55eb7e9db670 0x55eb7e9dda50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c93c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 0x55eb7e9cba70 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1109688495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c8d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3c6ff5700 0 client.25283 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1109688495 >> 172.21.15.93:0/1109688495 conn(0x55eb7e9b7890 msgr2=0x55eb7e9bcd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:56.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1109688495 shutdown_connections 2023-12-14T18:27:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.807+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1109688495 wait complete. 2023-12-14T18:27:56.815 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.808+0000 7fb3f5955b80 20 client.25283 trim_cache size 0 max 16384 2023-12-14T18:27:56.821 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:56.822 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:56.823 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:56.823 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:56.823 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.824+0000 7fb3f5955b80 1 Processor -- start 2023-12-14T18:27:56.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3f5955b80 1 -- start start 2023-12-14T18:27:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c8550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c8a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7e9ca370 con 0x55eb7e81e430 2023-12-14T18:27:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7e9c28c0 con 0x55eb7e9c1370 2023-12-14T18:27:56.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7e9be630 con 0x55eb7e9cc0f0 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3e37fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3e2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c8550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46888/0 (socket says 172.21.15.93:46888) 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3e2ffd700 1 -- 172.21.15.93:0/621451446 learned_addr learned my addr 172.21.15.93:0/621451446 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.825+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/621451446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3e37fe700 1 -- 172.21.15.93:0/621451446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9c1370 msgr2=0x55eb7e9c8a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3e37fe700 1 --2- 172.21.15.93:0/621451446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c8a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3e37fe700 1 -- 172.21.15.93:0/621451446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 msgr2=0x55eb7e9c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3e37fe700 1 --2- 172.21.15.93:0/621451446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3e37fe700 1 -- 172.21.15.93:0/621451446 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55eb7e8d9e60 con 0x55eb7e9cc0f0 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/621451446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3e37fe700 1 --2- 172.21.15.93:0/621451446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c3d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc011a60 tx=0x7fb3dc009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f4849d1e8abfd344 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3c9ffb700 1 -- 172.21.15.93:0/621451446 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc00e040 con 0x55eb7e9cc0f0 2023-12-14T18:27:56.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3c9ffb700 1 -- 172.21.15.93:0/621451446 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3dc00f040 con 0x55eb7e9cc0f0 2023-12-14T18:27:56.830 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.826+0000 7fb3c9ffb700 1 -- 172.21.15.93:0/621451446 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3dc0042b0 con 0x55eb7e9cc0f0 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 -- 172.21.15.93:0/621451446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 msgr2=0x55eb7e9c3d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/621451446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c3d10 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb3dc011a60 tx=0x7fb3dc009920 comp rx=0 tx=0).stop 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 -- 172.21.15.93:0/621451446 shutdown_connections 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/621451446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e81e430 0x55eb7e9c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/621451446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 0x55eb7e9c3d10 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/621451446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e9c1370 0x55eb7e9c8a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 -- 172.21.15.93:0/621451446 >> 172.21.15.93:0/621451446 conn(0x55eb7e9b7890 msgr2=0x55eb7e9d7e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 -- 172.21.15.93:0/621451446 shutdown_connections 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 -- 172.21.15.93:0/621451446 wait complete. 2023-12-14T18:27:56.833 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.827+0000 7fb3f5955b80 1 -- start start 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3f5955b80 1 Processor -- start 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 0x55eb7e9d0e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9c1370 0x55eb7e9d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3f5955b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 0x55eb7ea2e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7ea2ea10 con 0x55eb7e9c1370 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7ea2eb50 con 0x55eb7e81e430 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3f5955b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55eb7ea2ed10 con 0x55eb7e9cc0f0 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3f5955b80 10 client.0 _ll_register_callbacks cb 0x7fb3dc009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e27fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9c1370 0x55eb7e9d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e27fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9c1370 0x55eb7e9d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46892/0 (socket says 172.21.15.93:46892) 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e27fc700 1 -- 172.21.15.93:0/1084317727 learned_addr learned my addr 172.21.15.93:0/1084317727 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 0x55eb7e9d0e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e37fe700 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 0x55eb7ea2e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e27fc700 1 -- 172.21.15.93:0/1084317727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 msgr2=0x55eb7ea2e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:56.834 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 0x55eb7ea2e4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e27fc700 1 -- 172.21.15.93:0/1084317727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 msgr2=0x55eb7e9d0e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 0x55eb7e9d0e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.828+0000 7fb3e27fc700 1 -- 172.21.15.93:0/1084317727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55eb7e8da110 con 0x55eb7e9c1370 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.829+0000 7fb3e27fc700 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9c1370 0x55eb7e9d13a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80058e0 tx=0x7fb3d8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621f1c8248983c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.829+0000 7fb3c6ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.829+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1084317727 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3d8028040 con 0x55eb7e9c1370 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.829+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1084317727 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb3d80315c0 con 0x55eb7e9c1370 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.829+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1084317727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb3d409dc60 con 0x55eb7e9c1370 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.829+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1084317727 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb3d802a040 con 0x55eb7e9c1370 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.829+0000 7fb3f5955b80 10 client.15651 fetch_fsmap learned FSMap version 24 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.829+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1084317727 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x7fb3dc0117d0 con 0x55eb7e9c1370 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.829+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1084317727 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb3d409dc60 con 0x55eb7e9c1370 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1084317727 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7fb3d80309b0 con 0x55eb7e9c1370 2023-12-14T18:27:56.835 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3f5955b80 10 client.15651 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:27:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3f5955b80 10 client.15651 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:27:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3f5955b80 20 client.15651 populate_metadata read hostname 'smithi093' 2023-12-14T18:27:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55eb7ea30480 0x55eb7ea32890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:27:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3f5955b80 4 client.15651 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:27:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1084317727 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55eb7e90c9a0 con 0x55eb7ea30480 2023-12-14T18:27:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55eb7ea30480 0x55eb7ea32890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:27:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3c6ff5700 10 client.15651 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:56.836 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.830+0000 7fb3e2ffd700 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55eb7ea30480 0x55eb7ea32890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d97d9f3a26cea7b server_cookie=224efbf855c5642d in_seq=0 out_seq=0 2023-12-14T18:27:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.850+0000 7fb3c6ff5700 1 -- 172.21.15.93:0/1084317727 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55eb7e90c9a0 con 0x55eb7ea30480 2023-12-14T18:27:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.850+0000 7fb3c6ff5700 10 client.15651 handle_command_reply: tid=0 2023-12-14T18:27:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.850+0000 7fb3f5955b80 1 client.15651 shutdown 2023-12-14T18:27:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1084317727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9c1370 msgr2=0x55eb7e9d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9c1370 0x55eb7e9d13a0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7fb3d80058e0 tx=0x7fb3d8009310 comp rx=0 tx=0).stop 2023-12-14T18:27:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1084317727 shutdown_connections 2023-12-14T18:27:56.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55eb7ea30480 0x55eb7ea32890 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55eb7e9cc0f0 0x55eb7ea2e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55eb7e81e430 0x55eb7e9d0e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 --2- 172.21.15.93:0/1084317727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55eb7e9c1370 0x55eb7e9d13a0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3c6ff5700 0 client.15651 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1084317727 >> 172.21.15.93:0/1084317727 conn(0x55eb7e9b7890 msgr2=0x55eb7e9bc3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1084317727 shutdown_connections 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 1 -- 172.21.15.93:0/1084317727 wait complete. 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.851+0000 7fb3f5955b80 20 client.15651 trim_cache size 0 max 16384 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:27:56.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:27:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:27:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb3cc0a13e0 msgr2=0x7fb3cc0a3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb3cc0a13e0 0x7fb3cc0a3890 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb3d8008000 tx=0x7fb3d8003680 comp rx=0 tx=0).stop 2023-12-14T18:27:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e4100c50 msgr2=0x7fb3e419e5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:27:56.855 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e4100c50 0x7fb3e419e5d0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fb3d400d980 tx=0x7fb3d400dc90 comp rx=0 tx=0).stop 2023-12-14T18:27:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2956579943 shutdown_connections 2023-12-14T18:27:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb3cc0a13e0 0x7fb3cc0a3890 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb3e4100c50 0x7fb3e419e5d0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb3e4198640 0x7fb3e4198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 --2- 172.21.15.93:0/2956579943 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb3e4105560 0x7fb3e419eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:27:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2956579943 >> 172.21.15.93:0/2956579943 conn(0x7fb3e4077590 msgr2=0x7fb3e40fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:27:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2956579943 shutdown_connections 2023-12-14T18:27:56.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:27:56.852+0000 7fb3e97c7700 1 -- 172.21.15.93:0/2956579943 wait complete. 2023-12-14T18:27:57.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:56 smithi097 ceph-mon[43486]: pgmap v47: 129 pgs: 129 active+clean; 1.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-14T18:27:57.165 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:56 smithi093 ceph-mon[42070]: pgmap v47: 129 pgs: 129 active+clean; 1.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-14T18:27:57.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:56 smithi160 ceph-mon[43840]: pgmap v47: 129 pgs: 129 active+clean; 1.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-14T18:27:57.251 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:57.251 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:57.430 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:57.431 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:57.444 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:57.444 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:57.444 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 49 -lt 100 ']' 2023-12-14T18:27:57.444 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 49 + 1 2023-12-14T18:27:57.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=50 2023-12-14T18:27:57.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:57.502 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:57.502 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:57.503 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:57.503 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:57.503 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:58.393 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:58.393 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:58.588 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:58.589 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:58.603 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:58.603 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:58.603 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 50 -lt 100 ']' 2023-12-14T18:27:58.604 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 50 + 1 2023-12-14T18:27:58.632 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=51 2023-12-14T18:27:58.632 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:58.655 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:58.655 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:58.656 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:58.656 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:58.656 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:59.026 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:59.026 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:59.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:58 smithi097 ceph-mon[43486]: pgmap v48: 129 pgs: 129 active+clean; 1.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-14T18:27:59.180 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:59.180 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:59.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:59.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:59.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 51 -lt 100 ']' 2023-12-14T18:27:59.193 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 51 + 1 2023-12-14T18:27:59.215 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=52 2023-12-14T18:27:59.215 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:27:59.238 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:59.239 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:59.239 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:27:59.240 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:27:59.240 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:27:59.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:58 smithi160 ceph-mon[43840]: pgmap v48: 129 pgs: 129 active+clean; 1.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-14T18:27:59.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:58 smithi093 ceph-mon[42070]: pgmap v48: 129 pgs: 129 active+clean; 1.9 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 13 KiB/s wr, 6 op/s 2023-12-14T18:27:59.809 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:27:59.809 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:27:59.980 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:27:59.980 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:27:59.994 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:59.994 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:27:59.994 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 52 -lt 100 ']' 2023-12-14T18:27:59.994 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 52 + 1 2023-12-14T18:28:00.023 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=53 2023-12-14T18:28:00.023 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:00.052 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:00.052 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:00.053 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:00.053 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:00.053 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:00.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:27:59 smithi160 ceph-mon[43840]: mgrmap e25: x(active, since 91s), standbys: y, z 2023-12-14T18:28:00.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:27:59 smithi093 ceph-mon[42070]: mgrmap e25: x(active, since 91s), standbys: y, z 2023-12-14T18:28:00.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:27:59 smithi097 ceph-mon[43486]: mgrmap e25: x(active, since 91s), standbys: y, z 2023-12-14T18:28:00.467 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:00.468 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:00.630 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:00.630 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:00.643 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:00.643 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:00.643 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 53 -lt 100 ']' 2023-12-14T18:28:00.643 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 53 + 1 2023-12-14T18:28:00.673 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=54 2023-12-14T18:28:00.673 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:00.696 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:00.696 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:00.697 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:00.697 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:00.697 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:01.117 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:01.117 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:01.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:00 smithi160 ceph-mon[43840]: pgmap v49: 129 pgs: 129 active+clean; 2.0 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 16 op/s 2023-12-14T18:28:01.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:00 smithi093 ceph-mon[42070]: pgmap v49: 129 pgs: 129 active+clean; 2.0 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 16 op/s 2023-12-14T18:28:01.322 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:01.322 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:01.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:01.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:01.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 54 -lt 100 ']' 2023-12-14T18:28:01.335 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 54 + 1 2023-12-14T18:28:01.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:00 smithi097 ceph-mon[43486]: pgmap v49: 129 pgs: 129 active+clean; 2.0 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 26 KiB/s wr, 16 op/s 2023-12-14T18:28:01.373 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=55 2023-12-14T18:28:01.373 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:01.396 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:01.396 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:01.397 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:01.397 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:01.398 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:01.825 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:01.825 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:01.988 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:01.988 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:02.000 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:02.000 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:02.001 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 55 -lt 100 ']' 2023-12-14T18:28:02.001 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 55 + 1 2023-12-14T18:28:02.023 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=56 2023-12-14T18:28:02.023 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:02.051 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:02.051 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:02.052 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:02.052 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:02.052 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:02.600 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:02.600 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:02.788 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:02.788 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:02.802 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:02.802 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:02.802 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 56 -lt 100 ']' 2023-12-14T18:28:02.802 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 56 + 1 2023-12-14T18:28:02.831 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=57 2023-12-14T18:28:02.831 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:02.854 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:02.854 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:02.855 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:02.855 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:02.855 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:03.156 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:02 smithi093 ceph-mon[42070]: pgmap v50: 129 pgs: 129 active+clean; 2.0 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-12-14T18:28:03.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:02 smithi160 ceph-mon[43840]: pgmap v50: 129 pgs: 129 active+clean; 2.0 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-12-14T18:28:03.258 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:03.258 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:03.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:02 smithi097 ceph-mon[43486]: pgmap v50: 129 pgs: 129 active+clean; 2.0 MiB data, 84 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 13 op/s 2023-12-14T18:28:03.471 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:03.471 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:03.484 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:03.484 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:03.484 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 57 -lt 100 ']' 2023-12-14T18:28:03.484 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 57 + 1 2023-12-14T18:28:03.514 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=58 2023-12-14T18:28:03.514 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:03.538 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:03.538 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:03.539 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:03.539 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:03.539 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:04.033 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:04.033 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:04.246 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:04.246 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:04.259 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:04.260 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:04.260 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 58 -lt 100 ']' 2023-12-14T18:28:04.260 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 58 + 1 2023-12-14T18:28:04.289 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=59 2023-12-14T18:28:04.289 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:04.317 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:04.317 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:04.318 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:04.318 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:04.318 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:05.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:04 smithi160 ceph-mon[43840]: pgmap v51: 129 pgs: 129 active+clean; 2.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 47 KiB/s wr, 29 op/s 2023-12-14T18:28:05.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:04 smithi093 ceph-mon[42070]: pgmap v51: 129 pgs: 129 active+clean; 2.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 47 KiB/s wr, 29 op/s 2023-12-14T18:28:05.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:04 smithi097 ceph-mon[43486]: pgmap v51: 129 pgs: 129 active+clean; 2.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 47 KiB/s wr, 29 op/s 2023-12-14T18:28:05.483 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:05.483 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:05.696 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:05.696 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:05.718 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:05.718 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:05.718 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 59 -lt 100 ']' 2023-12-14T18:28:05.718 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 59 + 1 2023-12-14T18:28:05.747 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=60 2023-12-14T18:28:05.747 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:05.771 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:05.771 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:05.772 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:05.772 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:05.772 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:06.141 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:06.141 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:06.321 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:06.321 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:06.333 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:06.333 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:06.333 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 60 -lt 100 ']' 2023-12-14T18:28:06.333 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 60 + 1 2023-12-14T18:28:06.363 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=61 2023-12-14T18:28:06.364 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:06.387 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:06.387 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:06.388 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:06.389 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:06.389 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:07.008 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:07.008 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:07.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:06 smithi093 ceph-mon[42070]: pgmap v52: 129 pgs: 129 active+clean; 2.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-12-14T18:28:07.204 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:07.204 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:07.218 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:07.218 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:07.218 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 61 -lt 100 ']' 2023-12-14T18:28:07.218 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 61 + 1 2023-12-14T18:28:07.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:06 smithi160 ceph-mon[43840]: pgmap v52: 129 pgs: 129 active+clean; 2.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-12-14T18:28:07.250 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=62 2023-12-14T18:28:07.250 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:07.291 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:07.291 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:07.292 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:07.293 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:07.293 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:07.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:06 smithi097 ceph-mon[43486]: pgmap v52: 129 pgs: 129 active+clean; 2.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-12-14T18:28:07.641 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:07.641 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:07.754 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:07.754 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:07.767 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:07.767 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:07.767 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 62 -lt 100 ']' 2023-12-14T18:28:07.767 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 62 + 1 2023-12-14T18:28:07.797 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=63 2023-12-14T18:28:07.797 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:07.828 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:07.828 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:07.829 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:07.830 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:07.830 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:08.166 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:08.166 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:08.312 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:08.312 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:08.326 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:08.326 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:08.326 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 63 -lt 100 ']' 2023-12-14T18:28:08.326 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 63 + 1 2023-12-14T18:28:08.350 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=64 2023-12-14T18:28:08.350 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:08.370 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:08.370 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:08.372 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:08.372 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:08.372 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:08.733 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:08.733 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:08.949 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:08.949 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:08.975 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:08.975 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:08.975 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 64 -lt 100 ']' 2023-12-14T18:28:08.975 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 64 + 1 2023-12-14T18:28:09.005 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=65 2023-12-14T18:28:09.005 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:09.029 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:09.029 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:09.030 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:09.030 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:09.030 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:09.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:08 smithi093 ceph-mon[42070]: pgmap v53: 129 pgs: 129 active+clean; 2.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-12-14T18:28:09.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:08 smithi160 ceph-mon[43840]: pgmap v53: 129 pgs: 129 active+clean; 2.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-12-14T18:28:09.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:08 smithi097 ceph-mon[43486]: pgmap v53: 129 pgs: 129 active+clean; 2.1 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 40 KiB/s wr, 26 op/s 2023-12-14T18:28:09.465 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:09.466 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:09.628 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:09.629 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:09.643 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:09.643 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:09.644 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 65 -lt 100 ']' 2023-12-14T18:28:09.644 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 65 + 1 2023-12-14T18:28:09.680 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=66 2023-12-14T18:28:09.680 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:09.703 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:09.703 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:09.704 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:09.704 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:09.705 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:10.166 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:10.166 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:10.520 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:10.520 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:10.541 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:10.541 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:10.541 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 66 -lt 100 ']' 2023-12-14T18:28:10.542 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 66 + 1 2023-12-14T18:28:10.596 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=67 2023-12-14T18:28:10.596 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:10.651 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:10.651 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:10.652 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:10.652 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:10.652 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:11.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:10 smithi160 ceph-mon[43840]: pgmap v54: 129 pgs: 129 active+clean; 2.2 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 KiB/s wr, 29 op/s 2023-12-14T18:28:11.253 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:10 smithi093 ceph-mon[42070]: pgmap v54: 129 pgs: 129 active+clean; 2.2 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 KiB/s wr, 29 op/s 2023-12-14T18:28:11.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:10 smithi097 ceph-mon[43486]: pgmap v54: 129 pgs: 129 active+clean; 2.2 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 51 KiB/s wr, 29 op/s 2023-12-14T18:28:11.407 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:11.407 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:11.703 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:11.703 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:11.743 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:11.743 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:11.743 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 67 -lt 100 ']' 2023-12-14T18:28:11.743 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 67 + 1 2023-12-14T18:28:11.780 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=68 2023-12-14T18:28:11.780 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:12.192 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:12.192 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:12.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:12.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:12.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.883+0000 7f9ec5df8700 1 -- 172.21.15.93:0/3596674311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0100540 msgr2=0x7f9ec0100910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.883+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/3596674311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0100540 0x7f9ec0100910 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00099c0 tx=0x7f9eb0009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.883+0000 7f9ec5df8700 1 -- 172.21.15.93:0/3596674311 shutdown_connections 2023-12-14T18:28:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.883+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/3596674311 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0100540 0x7f9ec0100910 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.883+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/3596674311 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ec0105530 0x7f9ec010d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.883+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/3596674311 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0100ee0 0x7f9ec0104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:12.885 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.883+0000 7f9ec5df8700 1 -- 172.21.15.93:0/3596674311 >> 172.21.15.93:0/3596674311 conn(0x7f9ec0078210 msgr2=0x7f9ec0078610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.883+0000 7f9ec5df8700 1 -- 172.21.15.93:0/3596674311 shutdown_connections 2023-12-14T18:28:12.886 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.884+0000 7f9ec5df8700 1 -- 172.21.15.93:0/3596674311 wait complete. 2023-12-14T18:28:12.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.917+0000 7f9ec5df8700 1 Processor -- start 2023-12-14T18:28:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.917+0000 7f9ec5df8700 1 -- start start 2023-12-14T18:28:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ec5df8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ec0100540 0x7f9ec0196440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ec5df8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0100ee0 0x7f9ec0196980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:12.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ec5df8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0105530 0x7f9ec019ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ec5df8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec019b420 con 0x7f9ec0105530 2023-12-14T18:28:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ec5df8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec019b590 con 0x7f9ec0100ee0 2023-12-14T18:28:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ec5df8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ec019b750 con 0x7f9ec0100540 2023-12-14T18:28:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0105530 0x7f9ec019ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebf7fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ec0100540 0x7f9ec0196440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0105530 0x7f9ec019ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:53692/0 (socket says 172.21.15.93:53692) 2023-12-14T18:28:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebffff700 1 -- 172.21.15.93:0/2705129981 learned_addr learned my addr 172.21.15.93:0/2705129981 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0100ee0 0x7f9ec0196980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:12.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebffff700 1 -- 172.21.15.93:0/2705129981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ec0100540 msgr2=0x7f9ec0196440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebffff700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ec0100540 0x7f9ec0196440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebffff700 1 -- 172.21.15.93:0/2705129981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0100ee0 msgr2=0x7f9ec0196980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebffff700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0100ee0 0x7f9ec0196980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebffff700 1 -- 172.21.15.93:0/2705129981 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9eb00095f0 con 0x7f9ec0105530 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ec0100540 0x7f9ec0196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0100ee0 0x7f9ec0196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.918+0000 7f9ebffff700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0105530 0x7f9ec019ade0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800ef30 tx=0x7f9eb800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.919+0000 7f9ebcff9700 1 -- 172.21.15.93:0/2705129981 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb8018070 con 0x7f9ec0105530 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.919+0000 7f9ebcff9700 1 -- 172.21.15.93:0/2705129981 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9eb800f040 con 0x7f9ec0105530 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.919+0000 7f9ec5df8700 1 -- 172.21.15.93:0/2705129981 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ec0110340 con 0x7f9ec0105530 2023-12-14T18:28:12.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.919+0000 7f9ebcff9700 1 -- 172.21.15.93:0/2705129981 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb800cdb0 con 0x7f9ec0105530 2023-12-14T18:28:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.919+0000 7f9ec5df8700 1 -- 172.21.15.93:0/2705129981 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec01101b0 con 0x7f9ec0105530 2023-12-14T18:28:12.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.921+0000 7f9ebcff9700 1 -- 172.21.15.93:0/2705129981 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9eb8003a00 con 0x7f9ec0105530 2023-12-14T18:28:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.921+0000 7f9ebcff9700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9ea80a12c0 0x7f9ea80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.921+0000 7f9ebcff9700 1 -- 172.21.15.93:0/2705129981 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f9eb80ccba0 con 0x7f9ec0105530 2023-12-14T18:28:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.921+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9ea80a12c0 0x7f9ea80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:12.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:12.922+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9ea80a12c0 0x7f9ea80a3770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009990 tx=0x7f9eb001a040 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:12.924 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:12.924 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.023+0000 7f9ed1f86b80 1 Processor -- start 2023-12-14T18:28:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.023+0000 7f9ed1f86b80 1 -- start start 2023-12-14T18:28:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 0x55f66d2a9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2aa370 0x55f66d2aa7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d2b74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d194580 con 0x55f66d2aa370 2023-12-14T18:28:13.032 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d203810 con 0x55f66d2b50f0 2023-12-14T18:28:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d1fcca0 con 0x55f66d107430 2023-12-14T18:28:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2aa370 0x55f66d2aa7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebeffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2aa370 0x55f66d2aa7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:53706/0 (socket says 172.21.15.93:53706) 2023-12-14T18:28:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebeffd700 1 -- 172.21.15.93:0/3997306404 learned_addr learned my addr 172.21.15.93:0/3997306404 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 0x55f66d2a9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebffff700 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d2b74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebeffd700 1 -- 172.21.15.93:0/3997306404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 msgr2=0x55f66d2a9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 0x55f66d2a9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.033 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebeffd700 1 -- 172.21.15.93:0/3997306404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 msgr2=0x55f66d2b74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d2b74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebeffd700 1 -- 172.21.15.93:0/3997306404 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66d07e350 con 0x55f66d2aa370 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebffff700 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d2b74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.024+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2aa370 0x55f66d2aa7e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009d00 tx=0x7f9eb40116c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0ebf4ee228dfd96 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.025+0000 7f9ea5ffb700 1 -- 172.21.15.93:0/3997306404 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb4011e20 con 0x55f66d2aa370 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.025+0000 7f9ea5ffb700 1 -- 172.21.15.93:0/3997306404 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9eb4009e50 con 0x55f66d2aa370 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.025+0000 7f9ea5ffb700 1 -- 172.21.15.93:0/3997306404 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb400f7c0 con 0x55f66d2aa370 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/3997306404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2aa370 msgr2=0x55f66d2aa7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2aa370 0x55f66d2aa7e0 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f9eb4009d00 tx=0x7f9eb40116c0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/3997306404 shutdown_connections 2023-12-14T18:28:13.034 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d2b74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 0x55f66d2a9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/3997306404 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2aa370 0x55f66d2aa7e0 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/3997306404 >> 172.21.15.93:0/3997306404 conn(0x55f66d2a0890 msgr2=0x55f66d2a6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/3997306404 shutdown_connections 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/3997306404 wait complete. 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.026+0000 7f9ed1f86b80 1 -- start start 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ed1f86b80 1 Processor -- start 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 0x55f66d2b4a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2aa370 0x55f66d2b1d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2b50f0 0x55f66d2b2350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d198f30 con 0x55f66d2b50f0 2023-12-14T18:28:13.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d198bd0 con 0x55f66d2aa370 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d080640 con 0x55f66d107430 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ed1f86b80 10 client.0 _ll_register_callbacks cb 0x55f66d1bc8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2b50f0 0x55f66d2b2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2b50f0 0x55f66d2b2350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:53708/0 (socket says 172.21.15.93:53708) 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebffff700 1 -- 172.21.15.93:0/753231400 learned_addr learned my addr 172.21.15.93:0/753231400 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 0x55f66d2b4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2aa370 0x55f66d2b1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebffff700 1 -- 172.21.15.93:0/753231400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 msgr2=0x55f66d2b4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebffff700 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 0x55f66d2b4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebffff700 1 -- 172.21.15.93:0/753231400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2aa370 msgr2=0x55f66d2b1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebffff700 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2aa370 0x55f66d2b1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.027+0000 7f9ebffff700 1 -- 172.21.15.93:0/753231400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9eb4009930 con 0x55f66d2b50f0 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 0x55f66d2b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9ebffff700 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2b50f0 0x55f66d2b2350 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80999a0 tx=0x7f9eb8099460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=963791ec5bbe997e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9e9effd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9e9effd700 1 -- 172.21.15.93:0/753231400 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb809da40 con 0x55f66d2b50f0 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9e9effd700 1 -- 172.21.15.93:0/753231400 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f9eb80a1dd0 con 0x55f66d2b50f0 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/753231400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f66d29dea0 con 0x55f66d2b50f0 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9e9effd700 1 -- 172.21.15.93:0/753231400 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eb8098f00 con 0x55f66d2b50f0 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9ed1f86b80 10 client.15663 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:13.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/753231400 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f66d1c3110 con 0x55f66d2b50f0 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9e9effd700 1 -- 172.21.15.93:0/753231400 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f66d29dea0 con 0x55f66d2b50f0 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.028+0000 7f9e9effd700 1 -- 172.21.15.93:0/753231400 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9eb8098970 con 0x55f66d2b50f0 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.029+0000 7f9ed1f86b80 10 client.15663 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.029+0000 7f9ed1f86b80 10 client.15663 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.029+0000 7f9ed1f86b80 20 client.15663 populate_metadata read hostname 'smithi093' 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.029+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55f66d2c4a20 0x55f66d2c6e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.029+0000 7f9ed1f86b80 4 client.15663 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.029+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/753231400 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55f66d1f5c80 con 0x55f66d2c4a20 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.029+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55f66d2c4a20 0x55f66d2c6e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.029+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55f66d2c4a20 0x55f66d2c6e00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=faaf2411161934d6 server_cookie=ff2a5a44ee733ed1 in_seq=0 out_seq=0 2023-12-14T18:28:13.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.029+0000 7f9e9effd700 10 client.15663 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9e9effd700 1 -- 172.21.15.93:0/753231400 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55f66d1f5c80 con 0x55f66d2c4a20 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9e9effd700 10 client.15663 handle_command_reply: tid=0 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 client.15663 shutdown 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/753231400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2b50f0 msgr2=0x55f66d2b2350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2b50f0 0x55f66d2b2350 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f9eb80999a0 tx=0x7f9eb8099460 comp rx=0 tx=0).stop 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/753231400 shutdown_connections 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55f66d2c4a20 0x55f66d2c6e00 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d2b50f0 0x55f66d2b2350 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d107430 0x55f66d2b4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9e9effd700 0 client.15663 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:13.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/753231400 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2aa370 0x55f66d2b1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/753231400 >> 172.21.15.93:0/753231400 conn(0x55f66d2a0890 msgr2=0x55f66d2a5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/753231400 shutdown_connections 2023-12-14T18:28:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.030+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/753231400 wait complete. 2023-12-14T18:28:13.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.031+0000 7f9ed1f86b80 20 client.15663 trim_cache size 0 max 16384 2023-12-14T18:28:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.046+0000 7f9ed1f86b80 1 Processor -- start 2023-12-14T18:28:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.046+0000 7f9ed1f86b80 1 -- start start 2023-12-14T18:28:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d2bbcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 0x55f66d2b4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.053 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d2af020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d2aa740 con 0x55f66d107430 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d2a4fb0 con 0x55f66d2b50f0 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d2adc00 con 0x55f66d2aa370 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d2bbcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d2bbcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:53714/0 (socket says 172.21.15.93:53714) 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebf7fe700 1 -- 172.21.15.93:0/4109203895 learned_addr learned my addr 172.21.15.93:0/4109203895 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebffff700 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d2af020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 0x55f66d2b4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebf7fe700 1 -- 172.21.15.93:0/4109203895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 msgr2=0x55f66d2b4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:13.054 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 0x55f66d2b4580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebf7fe700 1 -- 172.21.15.93:0/4109203895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 msgr2=0x55f66d2af020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d2af020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebf7fe700 1 -- 172.21.15.93:0/4109203895 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f66d1c2e60 con 0x55f66d107430 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 0x55f66d2b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.047+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d2bbcc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00036c0 tx=0x7f9eb001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10fd848ca41c8931 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ea67fc700 1 -- 172.21.15.93:0/4109203895 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb002a030 con 0x55f66d107430 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ea67fc700 1 -- 172.21.15.93:0/4109203895 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9eb002b040 con 0x55f66d107430 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ea67fc700 1 -- 172.21.15.93:0/4109203895 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb0005460 con 0x55f66d107430 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/4109203895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 msgr2=0x55f66d2bbcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d2bbcc0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f9eb00036c0 tx=0x7f9eb001f000 comp rx=0 tx=0).stop 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/4109203895 shutdown_connections 2023-12-14T18:28:13.055 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d2bbcc0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d2af020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/4109203895 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 0x55f66d2b4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.048+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/4109203895 >> 172.21.15.93:0/4109203895 conn(0x55f66d2a0890 msgr2=0x55f66d2a5cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.049+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/4109203895 shutdown_connections 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.049+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/4109203895 wait complete. 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.049+0000 7f9ed1f86b80 1 -- start start 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.049+0000 7f9ed1f86b80 1 Processor -- start 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.049+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d30c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 0x55f66d30cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ed1f86b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d3178c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d30d580 con 0x55f66d107430 2023-12-14T18:28:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d30d6f0 con 0x55f66d2b50f0 2023-12-14T18:28:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ed1f86b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f66d317e80 con 0x55f66d2aa370 2023-12-14T18:28:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ed1f86b80 10 client.0 _ll_register_callbacks cb 0x7f9eb8099550 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d30c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebf7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d30c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:53726/0 (socket says 172.21.15.93:53726) 2023-12-14T18:28:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebf7fe700 1 -- 172.21.15.93:0/51838543 learned_addr learned my addr 172.21.15.93:0/51838543 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebffff700 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d3178c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 0x55f66d30cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebf7fe700 1 -- 172.21.15.93:0/51838543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 msgr2=0x55f66d30cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 0x55f66d30cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebf7fe700 1 -- 172.21.15.93:0/51838543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 msgr2=0x55f66d3178c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d3178c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebf7fe700 1 -- 172.21.15.93:0/51838543 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ec010d940 con 0x55f66d107430 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ebf7fe700 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d30c840 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009d10 tx=0x7f9eb0034b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80dc34d36eb18db6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9e9effd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9e9effd700 1 -- 172.21.15.93:0/51838543 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9eb002a040 con 0x55f66d107430 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9e9effd700 1 -- 172.21.15.93:0/51838543 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f9eb0034250 con 0x55f66d107430 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.050+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/51838543 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55f66d2aa200 con 0x55f66d107430 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9e9effd700 1 -- 172.21.15.93:0/51838543 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9eb0034410 con 0x55f66d107430 2023-12-14T18:28:13.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9ed1f86b80 10 client.15669 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/51838543 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x55f66d1a9940 con 0x55f66d107430 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9e9effd700 1 -- 172.21.15.93:0/51838543 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55f66d2aa200 con 0x55f66d107430 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9e9effd700 1 -- 172.21.15.93:0/51838543 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f9eb0024660 con 0x55f66d107430 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9ed1f86b80 10 client.15669 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9ed1f86b80 10 client.15669 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9ed1f86b80 20 client.15669 populate_metadata read hostname 'smithi093' 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55f66d319190 0x55f66d31b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9ed1f86b80 4 client.15669 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/51838543 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7f9eb4009e50 con 0x55f66d319190 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.051+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55f66d319190 0x55f66d31b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.052+0000 7f9e9effd700 10 client.15669 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:13.059 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.052+0000 7f9ebeffd700 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55f66d319190 0x55f66d31b5a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b24e2e3f0c465f4c server_cookie=7ba001fa9d45ca9c in_seq=0 out_seq=0 2023-12-14T18:28:13.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:12 smithi093 ceph-mon[42070]: pgmap v55: 129 pgs: 129 active+clean; 2.2 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 20 op/s 2023-12-14T18:28:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9e9effd700 1 -- 172.21.15.93:0/51838543 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7f9eb4009e50 con 0x55f66d319190 2023-12-14T18:28:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9e9effd700 10 client.15669 handle_command_reply: tid=0 2023-12-14T18:28:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 client.15669 shutdown 2023-12-14T18:28:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/51838543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 msgr2=0x55f66d30c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d30c840 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009d10 tx=0x7f9eb0034b40 comp rx=0 tx=0).stop 2023-12-14T18:28:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/51838543 shutdown_connections 2023-12-14T18:28:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55f66d319190 0x55f66d31b5a0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55f66d107430 0x55f66d30c840 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.074 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55f66d2b50f0 0x55f66d3178c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 --2- 172.21.15.93:0/51838543 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55f66d2aa370 0x55f66d30cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9e9effd700 0 client.15669 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/51838543 >> 172.21.15.93:0/51838543 conn(0x55f66d2a0890 msgr2=0x55f66d2c0460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/51838543 shutdown_connections 2023-12-14T18:28:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.072+0000 7f9ed1f86b80 1 -- 172.21.15.93:0/51838543 wait complete. 2023-12-14T18:28:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ed1f86b80 20 client.15669 trim_cache size 0 max 16384 2023-12-14T18:28:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:28:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:28:13.075 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:28:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:28:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 -- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9ea80a12c0 msgr2=0x7f9ea80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:13.076 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9ea80a12c0 0x7f9ea80a3770 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f9eb0009990 tx=0x7f9eb001a040 comp rx=0 tx=0).stop 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 -- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0105530 msgr2=0x7f9ec019ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0105530 0x7f9ec019ade0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9eb800ef30 tx=0x7f9eb800c3a0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 -- 172.21.15.93:0/2705129981 shutdown_connections 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9ea80a12c0 0x7f9ea80a3770 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ec0105530 0x7f9ec019ade0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ec0100540 0x7f9ec0196440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 --2- 172.21.15.93:0/2705129981 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ec0100ee0 0x7f9ec0196980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 -- 172.21.15.93:0/2705129981 >> 172.21.15.93:0/2705129981 conn(0x7f9ec0078210 msgr2=0x7f9ec00fddd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 -- 172.21.15.93:0/2705129981 shutdown_connections 2023-12-14T18:28:13.077 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:13.073+0000 7f9ec5df8700 1 -- 172.21.15.93:0/2705129981 wait complete. 2023-12-14T18:28:13.145 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:13.145 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:13.166 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:13.166 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:13.166 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 68 -lt 100 ']' 2023-12-14T18:28:13.166 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 68 + 1 2023-12-14T18:28:13.246 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=69 2023-12-14T18:28:13.246 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:13.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:12 smithi160 ceph-mon[43840]: pgmap v55: 129 pgs: 129 active+clean; 2.2 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 20 op/s 2023-12-14T18:28:13.283 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:13.283 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:13.285 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:13.285 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:13.285 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:13.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:12 smithi097 ceph-mon[43486]: pgmap v55: 129 pgs: 129 active+clean; 2.2 MiB data, 85 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 37 KiB/s wr, 20 op/s 2023-12-14T18:28:13.856 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:13.857 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:13.978 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:13.978 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:13.990 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:13.991 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:13.991 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 69 -lt 100 ']' 2023-12-14T18:28:13.991 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 69 + 1 2023-12-14T18:28:14.021 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=70 2023-12-14T18:28:14.021 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:14.049 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:14.049 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:14.050 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:14.050 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:14.050 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:14.482 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:14.482 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:14.636 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:14.636 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:14.650 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:14.650 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:14.650 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 70 -lt 100 ']' 2023-12-14T18:28:14.650 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 70 + 1 2023-12-14T18:28:14.680 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=71 2023-12-14T18:28:14.680 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:14.708 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:14.708 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:14.709 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:14.709 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:14.709 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:15.098 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:15.098 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:15.244 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:15.245 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:15.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:14 smithi160 ceph-mon[43840]: pgmap v56: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 53 KiB/s wr, 29 op/s 2023-12-14T18:28:15.259 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:15.259 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:15.259 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 71 -lt 100 ']' 2023-12-14T18:28:15.259 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 71 + 1 2023-12-14T18:28:15.288 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=72 2023-12-14T18:28:15.288 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:15.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:14 smithi093 ceph-mon[42070]: pgmap v56: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 53 KiB/s wr, 29 op/s 2023-12-14T18:28:15.334 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:15.334 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:15.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:15.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:15.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:15.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:14 smithi097 ceph-mon[43486]: pgmap v56: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 53 KiB/s wr, 29 op/s 2023-12-14T18:28:15.815 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:15.815 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:15.961 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:15.961 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:15.975 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:15.975 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:15.975 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 72 -lt 100 ']' 2023-12-14T18:28:15.975 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 72 + 1 2023-12-14T18:28:16.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=73 2023-12-14T18:28:16.004 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:16.033 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:16.033 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:16.034 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:16.034 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:16.034 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:16.398 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:16.398 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:16.536 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:16.536 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:16.548 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:16.548 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:16.549 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 73 -lt 100 ']' 2023-12-14T18:28:16.549 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 73 + 1 2023-12-14T18:28:16.571 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=74 2023-12-14T18:28:16.571 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:16.593 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:16.593 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:16.594 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:16.594 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:16.594 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:16.922 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:16.923 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:17.086 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:17.086 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:17.099 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:17.099 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:17.099 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 74 -lt 100 ']' 2023-12-14T18:28:17.099 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 74 + 1 2023-12-14T18:28:17.129 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=75 2023-12-14T18:28:17.129 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:17.156 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:17.156 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:17.157 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:17.158 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:17.158 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:17.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:16 smithi160 ceph-mon[43840]: pgmap v57: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-12-14T18:28:17.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:16 smithi093 ceph-mon[42070]: pgmap v57: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-12-14T18:28:17.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:16 smithi097 ceph-mon[43486]: pgmap v57: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-12-14T18:28:17.606 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:17.606 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:17.819 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:17.819 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:17.831 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:17.831 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:17.831 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 75 -lt 100 ']' 2023-12-14T18:28:17.832 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 75 + 1 2023-12-14T18:28:17.862 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=76 2023-12-14T18:28:17.862 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:17.884 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:17.885 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:17.886 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:17.886 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:17.886 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:18.322 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:18.323 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:18.519 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:18.519 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:18.532 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:18.532 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:18.532 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 76 -lt 100 ']' 2023-12-14T18:28:18.532 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 76 + 1 2023-12-14T18:28:18.561 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=77 2023-12-14T18:28:18.562 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:18.585 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:18.585 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:18.586 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:18.586 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:18.586 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:19.237 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:18 smithi093 ceph-mon[42070]: pgmap v58: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-12-14T18:28:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:18 smithi160 ceph-mon[43840]: pgmap v58: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-12-14T18:28:19.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:18 smithi097 ceph-mon[43486]: pgmap v58: 129 pgs: 129 active+clean; 2.3 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 26 KiB/s wr, 13 op/s 2023-12-14T18:28:19.397 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:19.397 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:19.577 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:19.577 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:19.592 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:19.592 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:19.592 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 77 -lt 100 ']' 2023-12-14T18:28:19.592 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 77 + 1 2023-12-14T18:28:19.670 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=78 2023-12-14T18:28:19.670 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:19.715 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:19.715 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:19.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:19.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:19.717 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:20.106 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:20.106 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:20.277 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:20.277 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:20.291 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:20.291 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:20.291 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 78 -lt 100 ']' 2023-12-14T18:28:20.291 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 78 + 1 2023-12-14T18:28:20.328 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=79 2023-12-14T18:28:20.329 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:20.356 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:20.357 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:20.358 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:20.358 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:20.358 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:20.872 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:20.872 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:21.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:20 smithi093 ceph-mon[42070]: pgmap v59: 129 pgs: 129 active+clean; 2.4 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 17 op/s 2023-12-14T18:28:21.102 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:21.102 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:21.118 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:21.118 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:21.118 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 79 -lt 100 ']' 2023-12-14T18:28:21.118 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 79 + 1 2023-12-14T18:28:21.145 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=80 2023-12-14T18:28:21.145 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:21.169 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:21.169 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:21.170 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:21.170 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:21.171 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:20 smithi160 ceph-mon[43840]: pgmap v59: 129 pgs: 129 active+clean; 2.4 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 17 op/s 2023-12-14T18:28:21.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:20 smithi097 ceph-mon[43486]: pgmap v59: 129 pgs: 129 active+clean; 2.4 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 33 KiB/s wr, 17 op/s 2023-12-14T18:28:21.714 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:21.714 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:21.977 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:21.977 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:21.989 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:21.989 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:21.990 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 80 -lt 100 ']' 2023-12-14T18:28:21.990 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 80 + 1 2023-12-14T18:28:22.011 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=81 2023-12-14T18:28:22.011 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:22.039 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:22.040 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:22.041 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:22.041 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:22.041 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:22.497 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:22.497 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:22.693 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:22.693 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:22.707 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:22.707 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:22.707 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 81 -lt 100 ']' 2023-12-14T18:28:22.707 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 81 + 1 2023-12-14T18:28:22.736 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=82 2023-12-14T18:28:22.736 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:22.765 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:22.765 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:22.767 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:22.767 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:22.767 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:23.158 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:22 smithi093 ceph-mon[42070]: pgmap v60: 129 pgs: 129 active+clean; 2.4 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-12-14T18:28:23.222 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:23.222 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:22 smithi160 ceph-mon[43840]: pgmap v60: 129 pgs: 129 active+clean; 2.4 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-12-14T18:28:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:22 smithi097 ceph-mon[43486]: pgmap v60: 129 pgs: 129 active+clean; 2.4 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 23 KiB/s wr, 13 op/s 2023-12-14T18:28:23.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:23.393 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:23.406 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:23.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:23.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 82 -lt 100 ']' 2023-12-14T18:28:23.407 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 82 + 1 2023-12-14T18:28:23.436 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=83 2023-12-14T18:28:23.436 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:23.465 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:23.465 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:23.466 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:23.466 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:23.466 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:24.039 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:24.039 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:24.235 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:24.235 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:24.248 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:24.248 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:24.249 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 83 -lt 100 ']' 2023-12-14T18:28:24.249 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 83 + 1 2023-12-14T18:28:24.277 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=84 2023-12-14T18:28:24.278 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:24.882 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:24.882 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:24.883 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:24.884 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:24.884 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:25.245 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:24 smithi093 ceph-mon[42070]: pgmap v61: 129 pgs: 129 active+clean; 2.5 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 17 op/s 2023-12-14T18:28:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:24 smithi160 ceph-mon[43840]: pgmap v61: 129 pgs: 129 active+clean; 2.5 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 17 op/s 2023-12-14T18:28:25.338 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:25.339 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:25.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:24 smithi097 ceph-mon[43486]: pgmap v61: 129 pgs: 129 active+clean; 2.5 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 32 KiB/s wr, 17 op/s 2023-12-14T18:28:25.493 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:25.493 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:25.508 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:25.508 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:25.508 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 84 -lt 100 ']' 2023-12-14T18:28:25.508 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 84 + 1 2023-12-14T18:28:25.536 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=85 2023-12-14T18:28:25.536 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:25.564 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:25.565 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:25.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:25.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:25.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:26.038 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:26.038 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:26.293 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:26.293 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:26.322 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:26.322 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:26.322 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 85 -lt 100 ']' 2023-12-14T18:28:26.322 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 85 + 1 2023-12-14T18:28:26.369 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=86 2023-12-14T18:28:26.369 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:26.405 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:26.406 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:26.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:26.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:26.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:27.022 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:27.022 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:27.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:26 smithi093 ceph-mon[42070]: pgmap v62: 129 pgs: 129 active+clean; 2.5 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-12-14T18:28:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:26 smithi160 ceph-mon[43840]: pgmap v62: 129 pgs: 129 active+clean; 2.5 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-12-14T18:28:27.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:26 smithi097 ceph-mon[43486]: pgmap v62: 129 pgs: 129 active+clean; 2.5 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-12-14T18:28:27.343 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:27.343 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:27.373 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:27.373 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:27.373 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 86 -lt 100 ']' 2023-12-14T18:28:27.374 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 86 + 1 2023-12-14T18:28:27.402 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=87 2023-12-14T18:28:27.402 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:27.447 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:27.447 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:27.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:27.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:27.449 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:28.038 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:28.038 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:28.192 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:28.193 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:28.206 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:28.206 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:28.206 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 87 -lt 100 ']' 2023-12-14T18:28:28.206 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 87 + 1 2023-12-14T18:28:28.268 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=88 2023-12-14T18:28:28.269 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:28.335 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:28.335 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:28.336 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:28.336 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:28.336 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:28.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.538+0000 7f5be37b8700 1 -- 172.21.15.93:0/3313124727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc100c50 msgr2=0x7f5bdc101020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.538+0000 7f5be37b8700 1 --2- 172.21.15.93:0/3313124727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc100c50 0x7f5bdc101020 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80099c0 tx=0x7f5bd8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.538+0000 7f5be37b8700 1 -- 172.21.15.93:0/3313124727 shutdown_connections 2023-12-14T18:28:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.538+0000 7f5be37b8700 1 --2- 172.21.15.93:0/3313124727 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc100c50 0x7f5bdc101020 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.538+0000 7f5be37b8700 1 --2- 172.21.15.93:0/3313124727 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bdc105560 0x7f5bdc10d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.538+0000 7f5be37b8700 1 --2- 172.21.15.93:0/3313124727 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc1015f0 0x7f5bdc104e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.538+0000 7f5be37b8700 1 -- 172.21.15.93:0/3313124727 >> 172.21.15.93:0/3313124727 conn(0x7f5bdc077560 msgr2=0x7f5bdc077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.539+0000 7f5be37b8700 1 -- 172.21.15.93:0/3313124727 shutdown_connections 2023-12-14T18:28:28.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.539+0000 7f5be37b8700 1 -- 172.21.15.93:0/3313124727 wait complete. 2023-12-14T18:28:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.539+0000 7f5be37b8700 1 Processor -- start 2023-12-14T18:28:28.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.539+0000 7f5be37b8700 1 -- start start 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be37b8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bdc100c50 0x7f5bdc072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be37b8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc1015f0 0x7f5bdc073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be37b8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc105560 0x7f5bdc1a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be37b8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc073b60 con 0x7f5bdc1015f0 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be37b8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc1a4b20 con 0x7f5bdc100c50 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be37b8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bdc1a4c60 con 0x7f5bdc105560 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be0d53700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc1015f0 0x7f5bdc073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be0d53700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc1015f0 0x7f5bdc073330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54706/0 (socket says 172.21.15.93:54706) 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be0d53700 1 -- 172.21.15.93:0/608885851 learned_addr learned my addr 172.21.15.93:0/608885851 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be1554700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bdc100c50 0x7f5bdc072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be1d55700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc105560 0x7f5bdc1a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be0d53700 1 -- 172.21.15.93:0/608885851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc105560 msgr2=0x7f5bdc1a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be0d53700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc105560 0x7f5bdc1a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be0d53700 1 -- 172.21.15.93:0/608885851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bdc100c50 msgr2=0x7f5bdc072df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:28.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be0d53700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bdc100c50 0x7f5bdc072df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be0d53700 1 -- 172.21.15.93:0/608885851 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bd80095f0 con 0x7f5bdc1015f0 2023-12-14T18:28:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be1d55700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc105560 0x7f5bdc1a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5be0d53700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc1015f0 0x7f5bdc073330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00e940 tx=0x7f5bcc00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.540+0000 7f5bd27fc700 1 -- 172.21.15.93:0/608885851 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc009730 con 0x7f5bdc1015f0 2023-12-14T18:28:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.541+0000 7f5bd27fc700 1 -- 172.21.15.93:0/608885851 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bcc00ee90 con 0x7f5bdc1015f0 2023-12-14T18:28:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.541+0000 7f5be37b8700 1 -- 172.21.15.93:0/608885851 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bdc110320 con 0x7f5bdc1015f0 2023-12-14T18:28:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.541+0000 7f5bd27fc700 1 -- 172.21.15.93:0/608885851 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc010ea0 con 0x7f5bdc1015f0 2023-12-14T18:28:28.545 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.541+0000 7f5be37b8700 1 -- 172.21.15.93:0/608885851 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc110190 con 0x7f5bdc1015f0 2023-12-14T18:28:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.543+0000 7f5bd27fc700 1 -- 172.21.15.93:0/608885851 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f5bcc00cbd0 con 0x7f5bdc1015f0 2023-12-14T18:28:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.543+0000 7f5bd27fc700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f5bc80a1340 0x7f5bc80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.544+0000 7f5be1554700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f5bc80a1340 0x7f5bc80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.544+0000 7f5bd27fc700 1 -- 172.21.15.93:0/608885851 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f5bcc014070 con 0x7f5bdc1015f0 2023-12-14T18:28:28.546 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.544+0000 7f5be1554700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f5bc80a1340 0x7f5bc80a37f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009990 tx=0x7f5bd8005fb0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.620+0000 7f5bef946b80 1 Processor -- start 2023-12-14T18:28:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.620+0000 7f5bef946b80 1 -- start start 2023-12-14T18:28:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.620+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 0x557a9c42de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.620+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c42e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.620+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c43b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.620+0000 7f5bef946b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c318580 con 0x557a9c4390f0 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.620+0000 7f5bef946b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c387810 con 0x557a9c28b430 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.620+0000 7f5bef946b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c380ca0 con 0x557a9c42e370 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1d55700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c43b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1d55700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c43b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54708/0 (socket says 172.21.15.93:54708) 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1d55700 1 -- 172.21.15.93:0/619260253 learned_addr learned my addr 172.21.15.93:0/619260253 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be0d53700 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c42e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1554700 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 0x557a9c42de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1d55700 1 -- 172.21.15.93:0/619260253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 msgr2=0x557a9c42e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1d55700 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c42e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1d55700 1 -- 172.21.15.93:0/619260253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 msgr2=0x557a9c42de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1d55700 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 0x557a9c42de30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1d55700 1 -- 172.21.15.93:0/619260253 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a9c202350 con 0x557a9c4390f0 2023-12-14T18:28:28.624 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1554700 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 0x557a9c42de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5be1d55700 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c43b4d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009db0 tx=0x7f5bd40077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403c1edf552e51f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5bc77fe700 1 -- 172.21.15.93:0/619260253 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4014610 con 0x557a9c4390f0 2023-12-14T18:28:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5bc77fe700 1 -- 172.21.15.93:0/619260253 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bd400f050 con 0x557a9c4390f0 2023-12-14T18:28:28.625 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.621+0000 7f5bc77fe700 1 -- 172.21.15.93:0/619260253 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4013870 con 0x557a9c4390f0 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 -- 172.21.15.93:0/619260253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 msgr2=0x557a9c43b4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c43b4d0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4009db0 tx=0x7f5bd40077e0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 -- 172.21.15.93:0/619260253 shutdown_connections 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c43b4d0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 0x557a9c42de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 --2- 172.21.15.93:0/619260253 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c42e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 -- 172.21.15.93:0/619260253 >> 172.21.15.93:0/619260253 conn(0x557a9c424890 msgr2=0x557a9c42a200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 -- 172.21.15.93:0/619260253 shutdown_connections 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 -- 172.21.15.93:0/619260253 wait complete. 2023-12-14T18:28:28.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.622+0000 7f5bef946b80 1 -- start start 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5bef946b80 1 Processor -- start 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 0x557a9c438940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c435c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4361a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5bef946b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c31cf30 con 0x557a9c4390f0 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5bef946b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c31cbd0 con 0x557a9c28b430 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5bef946b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c204640 con 0x557a9c42e370 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5bef946b80 10 client.0 _ll_register_callbacks cb 0x557a9c3408b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5be1d55700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4361a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5be1d55700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4361a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54724/0 (socket says 172.21.15.93:54724) 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5be1d55700 1 -- 172.21.15.93:0/2652222771 learned_addr learned my addr 172.21.15.93:0/2652222771 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5be1554700 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 0x557a9c438940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.623+0000 7f5be0d53700 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c435c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5be1d55700 1 -- 172.21.15.93:0/2652222771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 msgr2=0x557a9c435c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:28.630 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5be1d55700 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c435c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5be1d55700 1 -- 172.21.15.93:0/2652222771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 msgr2=0x557a9c438940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5be1d55700 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 0x557a9c438940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5be1d55700 1 -- 172.21.15.93:0/2652222771 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bd40099e0 con 0x557a9c4390f0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5be0d53700 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c435c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5be1d55700 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4361a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4007b30 tx=0x7f5bd4014b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c49a511f4c21aa1d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5bc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2652222771 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd4013550 con 0x557a9c4390f0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2652222771 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f5bd40136b0 con 0x557a9c4390f0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5bef946b80 1 -- 172.21.15.93:0/2652222771 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557a9c421ea0 con 0x557a9c4390f0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2652222771 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd4013870 con 0x557a9c4390f0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5bef946b80 10 client.15681 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5bef946b80 1 -- 172.21.15.93:0/2652222771 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557a9c347110 con 0x557a9c4390f0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.624+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2652222771 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557a9c421ea0 con 0x557a9c4390f0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2652222771 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bd4027860 con 0x557a9c4390f0 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5bef946b80 10 client.15681 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:28.631 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5bef946b80 10 client.15681 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5bef946b80 20 client.15681 populate_metadata read hostname 'smithi093' 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557a9c4489d0 0x557a9c44adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5bef946b80 4 client.15681 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5bef946b80 1 -- 172.21.15.93:0/2652222771 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557a9c379c80 con 0x557a9c4489d0 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5be1554700 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557a9c4489d0 0x557a9c44adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5be1554700 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557a9c4489d0 0x557a9c44adb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8265170c6c061ec3 server_cookie=620fbec4672d1d7c in_seq=0 out_seq=0 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.625+0000 7f5bc4ff9700 10 client.15681 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2652222771 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557a9c379c80 con 0x557a9c4489d0 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bc4ff9700 10 client.15681 handle_command_reply: tid=0 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bef946b80 1 client.15681 shutdown 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bef946b80 1 -- 172.21.15.93:0/2652222771 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 msgr2=0x557a9c4361a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4361a0 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4007b30 tx=0x7f5bd4014b10 comp rx=0 tx=0).stop 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bef946b80 1 -- 172.21.15.93:0/2652222771 shutdown_connections 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557a9c4489d0 0x557a9c44adb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4361a0 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.632 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c28b430 0x557a9c438940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2652222771 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c435c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bc4ff9700 0 client.15681 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.626+0000 7f5bef946b80 1 -- 172.21.15.93:0/2652222771 >> 172.21.15.93:0/2652222771 conn(0x557a9c424890 msgr2=0x557a9c429d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.627+0000 7f5bef946b80 1 -- 172.21.15.93:0/2652222771 shutdown_connections 2023-12-14T18:28:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.627+0000 7f5bef946b80 1 -- 172.21.15.93:0/2652222771 wait complete. 2023-12-14T18:28:28.633 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.627+0000 7f5bef946b80 20 client.15681 trim_cache size 0 max 16384 2023-12-14T18:28:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.641+0000 7f5bef946b80 1 Processor -- start 2023-12-14T18:28:28.644 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5bef946b80 1 -- start start 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c28b430 0x557a9c43fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c438570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c4390f0 0x557a9c429cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5bef946b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c42fd00 con 0x557a9c28b430 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5bef946b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c43ba70 con 0x557a9c4390f0 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5bef946b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c4341c0 con 0x557a9c42e370 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1554700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c28b430 0x557a9c43fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1554700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c28b430 0x557a9c43fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54740/0 (socket says 172.21.15.93:54740) 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1554700 1 -- 172.21.15.93:0/4087194366 learned_addr learned my addr 172.21.15.93:0/4087194366 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be0d53700 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c438570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1d55700 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c4390f0 0x557a9c429cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1554700 1 -- 172.21.15.93:0/4087194366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 msgr2=0x557a9c438570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1554700 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c438570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1554700 1 -- 172.21.15.93:0/4087194366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c4390f0 msgr2=0x557a9c429cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:28.645 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1554700 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c4390f0 0x557a9c429cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1554700 1 -- 172.21.15.93:0/4087194366 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557a9c346e60 con 0x557a9c28b430 2023-12-14T18:28:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1d55700 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c4390f0 0x557a9c429cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:28:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5be1554700 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c28b430 0x557a9c43fcf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80036c0 tx=0x7f5bd8022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc7f2b624ff964a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.642+0000 7f5bc7fff700 1 -- 172.21.15.93:0/4087194366 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd80325b0 con 0x557a9c28b430 2023-12-14T18:28:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bc7fff700 1 -- 172.21.15.93:0/4087194366 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bd802d040 con 0x557a9c28b430 2023-12-14T18:28:28.646 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bc7fff700 1 -- 172.21.15.93:0/4087194366 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd8004350 con 0x557a9c28b430 2023-12-14T18:28:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bef946b80 1 -- 172.21.15.93:0/4087194366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c28b430 msgr2=0x557a9c43fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bef946b80 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c28b430 0x557a9c43fcf0 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f5bd80036c0 tx=0x7f5bd8022000 comp rx=0 tx=0).stop 2023-12-14T18:28:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bef946b80 1 -- 172.21.15.93:0/4087194366 shutdown_connections 2023-12-14T18:28:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bef946b80 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c28b430 0x557a9c43fcf0 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bef946b80 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c4390f0 0x557a9c429cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bef946b80 1 --2- 172.21.15.93:0/4087194366 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c42e370 0x557a9c438570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bef946b80 1 -- 172.21.15.93:0/4087194366 >> 172.21.15.93:0/4087194366 conn(0x557a9c424890 msgr2=0x557a9c444e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bef946b80 1 -- 172.21.15.93:0/4087194366 shutdown_connections 2023-12-14T18:28:28.649 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.643+0000 7f5bef946b80 1 -- 172.21.15.93:0/4087194366 wait complete. 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5bef946b80 1 -- start start 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5bef946b80 1 Processor -- start 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c28b430 0x557a9c4951c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c42e370 0x557a9c495700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5bef946b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4907f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5bef946b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c490e40 con 0x557a9c4390f0 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5bef946b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c490fb0 con 0x557a9c42e370 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5bef946b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557a9c491170 con 0x557a9c28b430 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5bef946b80 10 client.0 _ll_register_callbacks cb 0x7f5bcc095bd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5be1d55700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4907f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5be1d55700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4907f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:54750/0 (socket says 172.21.15.93:54750) 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5be1d55700 1 -- 172.21.15.93:0/2467230864 learned_addr learned my addr 172.21.15.93:0/2467230864 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5be0d53700 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c42e370 0x557a9c495700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.644+0000 7f5be1554700 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c28b430 0x557a9c4951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5be1d55700 1 -- 172.21.15.93:0/2467230864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c28b430 msgr2=0x557a9c4951c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:28.650 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5be1d55700 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c28b430 0x557a9c4951c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5be1d55700 1 -- 172.21.15.93:0/2467230864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c42e370 msgr2=0x557a9c495700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5be1d55700 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c42e370 0x557a9c495700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5be1d55700 1 -- 172.21.15.93:0/2467230864 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bdc10d900 con 0x557a9c4390f0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5be1d55700 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4907f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4007820 tx=0x7f5bd4027e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0a23fa4157b1a5f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5bc4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2467230864 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bd40060a0 con 0x557a9c4390f0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2467230864 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f5bd40092d0 con 0x557a9c4390f0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5bef946b80 1 -- 172.21.15.93:0/2467230864 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f5bd400f050 con 0x557a9c4390f0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2467230864 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5bd4004ef0 con 0x557a9c4390f0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5bef946b80 10 client.15687 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5bef946b80 1 -- 172.21.15.93:0/2467230864 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x557a9c32d940 con 0x557a9c4390f0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2467230864 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5bd400f050 con 0x557a9c4390f0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.645+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2467230864 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f5bd401c8a0 con 0x557a9c4390f0 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.646+0000 7f5bef946b80 10 client.15687 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.646+0000 7f5bef946b80 10 client.15687 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:28:28.651 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.646+0000 7f5bef946b80 20 client.15687 populate_metadata read hostname 'smithi093' 2023-12-14T18:28:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.646+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557a9c49de80 0x557a9c4a0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.646+0000 7f5bef946b80 4 client.15687 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:28:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.646+0000 7f5bef946b80 1 -- 172.21.15.93:0/2467230864 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557a9c3799a0 con 0x557a9c49de80 2023-12-14T18:28:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.646+0000 7f5be1554700 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557a9c49de80 0x557a9c4a0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.646+0000 7f5be1554700 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557a9c49de80 0x557a9c4a0290 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9504ac2d7777720d server_cookie=4d841a4381a1781f in_seq=0 out_seq=0 2023-12-14T18:28:28.652 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.646+0000 7f5bc4ff9700 10 client.15687 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bc4ff9700 1 -- 172.21.15.93:0/2467230864 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557a9c3799a0 con 0x557a9c49de80 2023-12-14T18:28:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bc4ff9700 10 client.15687 handle_command_reply: tid=0 2023-12-14T18:28:28.670 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 client.15687 shutdown 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 -- 172.21.15.93:0/2467230864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 msgr2=0x557a9c4907f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4907f0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f5bd4007820 tx=0x7f5bd4027e10 comp rx=0 tx=0).stop 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 -- 172.21.15.93:0/2467230864 shutdown_connections 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557a9c49de80 0x557a9c4a0290 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557a9c4390f0 0x557a9c4907f0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bc4ff9700 0 client.15687 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557a9c28b430 0x557a9c4951c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 --2- 172.21.15.93:0/2467230864 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557a9c42e370 0x557a9c495700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 -- 172.21.15.93:0/2467230864 >> 172.21.15.93:0/2467230864 conn(0x557a9c424890 msgr2=0x557a9c429cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 -- 172.21.15.93:0/2467230864 shutdown_connections 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 1 -- 172.21.15.93:0/2467230864 wait complete. 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.668+0000 7f5bef946b80 20 client.15687 trim_cache size 0 max 16384 2023-12-14T18:28:28.671 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:28:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:28:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:28:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:28:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 -- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f5bc80a1340 msgr2=0x7f5bc80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:28.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f5bc80a1340 0x7f5bc80a37f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5bd8009990 tx=0x7f5bd8005fb0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 -- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc1015f0 msgr2=0x7f5bdc073330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc1015f0 0x7f5bdc073330 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00e940 tx=0x7f5bcc00ec50 comp rx=0 tx=0).stop 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 -- 172.21.15.93:0/608885851 shutdown_connections 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5bdc105560 0x7f5bdc1a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5bdc100c50 0x7f5bdc072df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f5bc80a1340 0x7f5bc80a37f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 --2- 172.21.15.93:0/608885851 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5bdc1015f0 0x7f5bdc073330 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 -- 172.21.15.93:0/608885851 >> 172.21.15.93:0/608885851 conn(0x7f5bdc077560 msgr2=0x7f5bdc103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 -- 172.21.15.93:0/608885851 shutdown_connections 2023-12-14T18:28:28.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:28.669+0000 7f5be37b8700 1 -- 172.21.15.93:0/608885851 wait complete. 2023-12-14T18:28:28.912 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:28.913 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:29.092 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:29.092 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:29.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:29.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:29.114 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 88 -lt 100 ']' 2023-12-14T18:28:29.114 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 88 + 1 2023-12-14T18:28:29.143 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=89 2023-12-14T18:28:29.144 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:29.175 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:29.175 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:29.176 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:29.176 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:29.176 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:28 smithi160 ceph-mon[43840]: pgmap v63: 129 pgs: 129 active+clean; 2.5 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-12-14T18:28:29.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:28 smithi093 ceph-mon[42070]: pgmap v63: 129 pgs: 129 active+clean; 2.5 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-12-14T18:28:29.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:28 smithi097 ceph-mon[43486]: pgmap v63: 129 pgs: 129 active+clean; 2.5 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 16 KiB/s wr, 7 op/s 2023-12-14T18:28:29.579 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:29.579 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:29.742 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:29.742 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:29.755 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:29.755 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:29.755 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 89 -lt 100 ']' 2023-12-14T18:28:29.755 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 89 + 1 2023-12-14T18:28:29.777 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=90 2023-12-14T18:28:29.777 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:29.805 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:29.805 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:29.806 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:29.806 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:29.806 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:30.238 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:30.238 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:30.501 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:30.501 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:30.514 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:30.514 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:30.514 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 90 -lt 100 ']' 2023-12-14T18:28:30.514 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 90 + 1 2023-12-14T18:28:30.560 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=91 2023-12-14T18:28:30.560 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:30.583 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:30.583 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:30.584 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:30.585 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:30.585 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:31.045 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:31.046 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:31.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:30 smithi160 ceph-mon[43840]: pgmap v64: 129 pgs: 129 active+clean; 2.7 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 KiB/s wr, 16 op/s 2023-12-14T18:28:31.284 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:31.284 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:31.296 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:31.296 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:31.297 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 91 -lt 100 ']' 2023-12-14T18:28:31.297 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 91 + 1 2023-12-14T18:28:31.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:30 smithi093 ceph-mon[42070]: pgmap v64: 129 pgs: 129 active+clean; 2.7 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 KiB/s wr, 16 op/s 2023-12-14T18:28:31.327 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=92 2023-12-14T18:28:31.327 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:31.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:30 smithi097 ceph-mon[43486]: pgmap v64: 129 pgs: 129 active+clean; 2.7 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 32 KiB/s wr, 16 op/s 2023-12-14T18:28:31.354 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:31.354 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:31.355 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:31.355 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:31.355 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:32.229 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:32.229 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:32.475 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:32.476 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:32.489 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:32.489 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:32.489 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 92 -lt 100 ']' 2023-12-14T18:28:32.489 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 92 + 1 2023-12-14T18:28:32.518 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=93 2023-12-14T18:28:32.518 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:32.564 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:32.564 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:32.565 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:32.565 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:32.566 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:33.020 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:33.021 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:33.183 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:33.184 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:33.196 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:33.196 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:33.196 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 93 -lt 100 ']' 2023-12-14T18:28:33.196 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 93 + 1 2023-12-14T18:28:33.235 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=94 2023-12-14T18:28:33.235 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:33.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:32 smithi160 ceph-mon[43840]: pgmap v65: 129 pgs: 129 active+clean; 2.7 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-12-14T18:28:33.264 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:33.264 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:33.265 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:33.265 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:33.265 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:32 smithi093 ceph-mon[42070]: pgmap v65: 129 pgs: 129 active+clean; 2.7 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-12-14T18:28:33.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:32 smithi097 ceph-mon[43486]: pgmap v65: 129 pgs: 129 active+clean; 2.7 MiB data, 86 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 26 KiB/s wr, 12 op/s 2023-12-14T18:28:33.771 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:33.771 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:33.925 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:33.925 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:33.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:33.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:33.939 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 94 -lt 100 ']' 2023-12-14T18:28:33.939 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 94 + 1 2023-12-14T18:28:33.968 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=95 2023-12-14T18:28:33.968 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:33.999 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:34.000 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:34.001 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:34.001 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:34.001 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:34.396 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:34.396 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:34.516 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:34.517 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:34.529 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:34.529 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:34.529 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 95 -lt 100 ']' 2023-12-14T18:28:34.529 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 95 + 1 2023-12-14T18:28:34.551 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=96 2023-12-14T18:28:34.551 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:34.575 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:34.575 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:34.576 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:34.576 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:34.576 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:34.987 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:34.987 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:35.108 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:35.108 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:35.122 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:35.122 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:35.122 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 96 -lt 100 ']' 2023-12-14T18:28:35.122 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 96 + 1 2023-12-14T18:28:35.151 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=97 2023-12-14T18:28:35.151 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:35.175 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:35.175 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:35.176 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:35.176 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:35.176 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:35.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:34 smithi160 ceph-mon[43840]: pgmap v66: 129 pgs: 129 active+clean; 2.7 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-12-14T18:28:35.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:34 smithi093 ceph-mon[42070]: pgmap v66: 129 pgs: 129 active+clean; 2.7 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-12-14T18:28:35.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:34 smithi097 ceph-mon[43486]: pgmap v66: 129 pgs: 129 active+clean; 2.7 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2023-12-14T18:28:35.512 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:35.512 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:35.650 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:35.650 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:35.663 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:35.663 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:35.663 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 97 -lt 100 ']' 2023-12-14T18:28:35.663 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 97 + 1 2023-12-14T18:28:35.684 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=98 2023-12-14T18:28:35.684 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:35.712 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:35.712 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:35.713 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:35.714 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:35.714 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:36.162 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:36.162 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:36.299 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:36.299 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:36.313 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:36.313 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:36.313 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 98 -lt 100 ']' 2023-12-14T18:28:36.313 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 98 + 1 2023-12-14T18:28:36.342 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=99 2023-12-14T18:28:36.343 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:36.371 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:36.371 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:36.372 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:36.372 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:36.372 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:36.828 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:36.828 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:37.025 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:37.025 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:37.045 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:37.045 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:37.045 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 99 -lt 100 ']' 2023-12-14T18:28:37.045 INFO:tasks.workunit.client.0.smithi093.stderr:++ expr 99 + 1 2023-12-14T18:28:37.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:36 smithi093 ceph-mon[42070]: pgmap v67: 129 pgs: 129 active+clean; 2.7 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-12-14T18:28:37.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:36 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:28:37.092 INFO:tasks.workunit.client.0.smithi093.stderr:+ c=100 2023-12-14T18:28:37.092 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:37.097 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:37.097 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:37.098 INFO:tasks.workunit.client.0.smithi093.stderr:+ c1=2 2023-12-14T18:28:37.098 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo 3 2023-12-14T18:28:37.098 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo tee /proc/sys/vm/drop_caches 2023-12-14T18:28:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:36 smithi160 ceph-mon[43840]: pgmap v67: 129 pgs: 129 active+clean; 2.7 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-12-14T18:28:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:36 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:28:37.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:36 smithi097 ceph-mon[43486]: pgmap v67: 129 pgs: 129 active+clean; 2.7 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-12-14T18:28:37.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:36 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:28:37.978 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfacl d1 2023-12-14T18:28:37.978 INFO:tasks.workunit.client.0.smithi093.stderr:++ grep -c nobody:rw 2023-12-14T18:28:38.241 INFO:tasks.workunit.client.0.smithi093.stderr:+ c2=2 2023-12-14T18:28:38.241 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:38.257 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:38.257 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 2 -ne 2 ']' 2023-12-14T18:28:38.257 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 100 -lt 100 ']' 2023-12-14T18:28:38.268 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir d1 2023-12-14T18:28:38.304 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n system.posix_acl_access -v 0x02000000 d1 2023-12-14T18:28:38.309 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n system.posix_acl_default -v 0x02000000 . 2023-12-14T18:28:38.312 INFO:tasks.workunit.client.0.smithi093.stderr:+ expect_failure getfattr -n system.posix_acl_access d1 2023-12-14T18:28:38.312 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n system.posix_acl_access d1 2023-12-14T18:28:38.336 INFO:tasks.workunit.client.0.smithi093.stderr:d1: system.posix_acl_access: No such attribute 2023-12-14T18:28:38.337 INFO:tasks.workunit.client.0.smithi093.stderr:+ return 0 2023-12-14T18:28:38.337 INFO:tasks.workunit.client.0.smithi093.stderr:+ expect_failure getfattr -n system.posix_acl_default . 2023-12-14T18:28:38.337 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n system.posix_acl_default . 2023-12-14T18:28:38.337 INFO:tasks.workunit.client.0.smithi093.stderr:.: system.posix_acl_default: No such attribute 2023-12-14T18:28:38.337 INFO:tasks.workunit.client.0.smithi093.stderr:+ return 0 2023-12-14T18:28:38.337 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir d1 2023-12-14T18:28:38.340 INFO:tasks.workunit.client.0.smithi093.stderr:+ cd .. 2023-12-14T18:28:38.340 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir testdir 2023-12-14T18:28:38.352 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo OK 2023-12-14T18:28:38.353 INFO:tasks.workunit.client.0.smithi093.stdout:OK 2023-12-14T18:28:38.354 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:28:38.354 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:28:38.888 INFO:tasks.workunit:Running workunit fs/misc/chmod.sh... 2023-12-14T18:28:38.888 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/chmod.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh 2023-12-14T18:28:39.011 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-12-14T18:28:39.011 INFO:tasks.workunit.client.0.smithi093.stderr:+ file=test_chmod.96319 2023-12-14T18:28:39.012 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo foo 2023-12-14T18:28:39.015 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-12-14T18:28:39.015 INFO:tasks.workunit.client.0.smithi093.stderr:+ chmod 400 test_chmod.96319 2023-12-14T18:28:39.042 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-12-14T18:28:39.042 INFO:tasks.workunit.client.0.smithi093.stderr:+ check_perms test_chmod.96319 -r-------- 2023-12-14T18:28:39.043 INFO:tasks.workunit.client.0.smithi093.stderr:+ file=test_chmod.96319 2023-12-14T18:28:39.043 INFO:tasks.workunit.client.0.smithi093.stderr:++ ls -la test_chmod.96319 2023-12-14T18:28:39.066 INFO:tasks.workunit.client.0.smithi093.stderr:+ r='-r--------. 1 ubuntu ubuntu 4 Dec 14 18:28 test_chmod.96319' 2023-12-14T18:28:39.066 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-12-14T18:28:39.066 INFO:tasks.workunit.client.0.smithi093.stderr:+ perms=-r-------- 2023-12-14T18:28:39.066 INFO:tasks.workunit.client.0.smithi093.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Dec 14 18:28 test_chmod.96319 2023-12-14T18:28:39.066 INFO:tasks.workunit.client.0.smithi093.stderr:++ awk '{print $1}' 2023-12-14T18:28:39.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:38 smithi160 ceph-mon[43840]: pgmap v68: 129 pgs: 129 active+clean; 2.7 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-12-14T18:28:39.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:38 smithi093 ceph-mon[42070]: pgmap v68: 129 pgs: 129 active+clean; 2.7 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-12-14T18:28:39.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:38 smithi097 ceph-mon[43486]: pgmap v68: 129 pgs: 129 active+clean; 2.7 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 11 op/s 2023-12-14T18:28:39.345 INFO:tasks.workunit.client.0.smithi093.stderr:+ test -r-------- '!=' -r--------. 2023-12-14T18:28:39.345 INFO:tasks.workunit.client.0.smithi093.stderr:++ echo -r--------. 1 ubuntu ubuntu 4 Dec 14 18:28 test_chmod.96319 2023-12-14T18:28:39.345 INFO:tasks.workunit.client.0.smithi093.stderr:++ awk '{print $1}' 2023-12-14T18:28:39.347 INFO:tasks.workunit.client.0.smithi093.stderr:+ test -r--------. '!=' -r--------. 2023-12-14T18:28:39.348 INFO:tasks.workunit.client.0.smithi093.stderr:+ set +e 2023-12-14T18:28:39.348 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo bar 2023-12-14T18:28:39.348 INFO:tasks.workunit.client.0.smithi093.stderr:/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/chmod.sh: line 40: test_chmod.96319: Permission denied 2023-12-14T18:28:39.348 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 1 = 0 2023-12-14T18:28:39.348 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-12-14T18:28:39.348 INFO:tasks.workunit.client.0.smithi093.stderr:+ chmod 600 test_chmod.96319 2023-12-14T18:28:39.349 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo bar 2023-12-14T18:28:39.349 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-12-14T18:28:39.349 INFO:tasks.workunit.client.0.smithi093.stderr:+ check_perms test_chmod.96319 -rw------- 2023-12-14T18:28:39.349 INFO:tasks.workunit.client.0.smithi093.stderr:+ file=test_chmod.96319 2023-12-14T18:28:39.349 INFO:tasks.workunit.client.0.smithi093.stderr:++ ls -la test_chmod.96319 2023-12-14T18:28:39.350 INFO:tasks.workunit.client.0.smithi093.stderr:+ r='-rw-------. 1 ubuntu ubuntu 8 Dec 14 18:28 test_chmod.96319' 2023-12-14T18:28:39.350 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-12-14T18:28:39.350 INFO:tasks.workunit.client.0.smithi093.stderr:+ perms=-rw------- 2023-12-14T18:28:39.350 INFO:tasks.workunit.client.0.smithi093.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Dec 14 18:28 test_chmod.96319 2023-12-14T18:28:39.350 INFO:tasks.workunit.client.0.smithi093.stderr:++ awk '{print $1}' 2023-12-14T18:28:39.351 INFO:tasks.workunit.client.0.smithi093.stderr:+ test -rw------- '!=' -rw-------. 2023-12-14T18:28:39.352 INFO:tasks.workunit.client.0.smithi093.stderr:++ echo -rw-------. 1 ubuntu ubuntu 8 Dec 14 18:28 test_chmod.96319 2023-12-14T18:28:39.352 INFO:tasks.workunit.client.0.smithi093.stderr:++ awk '{print $1}' 2023-12-14T18:28:39.354 INFO:tasks.workunit.client.0.smithi093.stderr:+ test -rw-------. '!=' -rw-------. 2023-12-14T18:28:39.354 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo foo 2023-12-14T18:28:39.354 INFO:tasks.workunit.client.0.smithi093.stderr:+ test 0 '!=' 0 2023-12-14T18:28:39.355 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:28:39.355 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:28:39.399 INFO:tasks.workunit:Running workunit fs/misc/dac_override.sh... 2023-12-14T18:28:39.399 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/dac_override.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dac_override.sh 2023-12-14T18:28:39.428 INFO:tasks.workunit.client.0.smithi093.stderr:+ set -e 2023-12-14T18:28:39.428 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir -p testdir 2023-12-14T18:28:39.430 INFO:tasks.workunit.client.0.smithi093.stderr:+ file=test_chmod.96383 2023-12-14T18:28:39.430 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo foo 2023-12-14T18:28:39.431 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo chmod 600 testdir 2023-12-14T18:28:39.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ expect_failure cat testdir/test_chmod.96383 2023-12-14T18:28:39.448 INFO:tasks.workunit.client.0.smithi093.stderr:+ cat testdir/test_chmod.96383 2023-12-14T18:28:39.466 INFO:tasks.workunit.client.0.smithi093.stderr:cat: testdir/test_chmod.96383: Permission denied 2023-12-14T18:28:39.466 INFO:tasks.workunit.client.0.smithi093.stderr:+ return 0 2023-12-14T18:28:39.466 INFO:tasks.workunit.client.0.smithi093.stderr:+ sudo cat testdir/test_chmod.96383 2023-12-14T18:28:39.476 INFO:tasks.workunit.client.0.smithi093.stdout:foo 2023-12-14T18:28:39.477 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:28:39.478 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:28:39.564 INFO:tasks.workunit:Running workunit fs/misc/direct_io.py... 2023-12-14T18:28:39.564 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/direct_io.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/direct_io.py 2023-12-14T18:28:40.480 INFO:tasks.workunit.client.0.smithi093.stdout:ok 2023-12-14T18:28:40.485 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:28:40.485 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:28:40.528 INFO:tasks.workunit:Running workunit fs/misc/dirfrag.sh... 2023-12-14T18:28:40.528 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/dirfrag.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/dirfrag.sh 2023-12-14T18:28:40.618 INFO:tasks.workunit.client.0.smithi093.stdout:creating folder hierarchy 2023-12-14T18:28:41.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:40 smithi160 ceph-mon[43840]: pgmap v69: 129 pgs: 129 active+clean; 2.9 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-12-14T18:28:41.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:40 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:28:41.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:40 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:28:41.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:40 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:28:41.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:40 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:28:41.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:40 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:28:41.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:40 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:28:41.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:40 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:28:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:40 smithi093 ceph-mon[42070]: pgmap v69: 129 pgs: 129 active+clean; 2.9 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-12-14T18:28:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:40 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:28:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:40 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:28:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:40 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:28:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:40 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:28:41.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:40 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:28:41.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:40 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:28:41.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:40 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:28:41.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:40 smithi097 ceph-mon[43486]: pgmap v69: 129 pgs: 129 active+clean; 2.9 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 36 KiB/s wr, 15 op/s 2023-12-14T18:28:41.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:40 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:28:41.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:40 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:28:41.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:40 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:28:41.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:40 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:28:41.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:40 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:28:41.346 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:40 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:28:41.346 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:40 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:28:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.958+0000 7f8c31331700 1 -- 172.21.15.93:0/623719986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c106350 msgr2=0x7f8c2c106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:41.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.958+0000 7f8c31331700 1 --2- 172.21.15.93:0/623719986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c106350 0x7f8c2c106720 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c009990 tx=0x7f8c1c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:28:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.958+0000 7f8c31331700 1 -- 172.21.15.93:0/623719986 shutdown_connections 2023-12-14T18:28:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.958+0000 7f8c31331700 1 --2- 172.21.15.93:0/623719986 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c106350 0x7f8c2c106720 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.958+0000 7f8c31331700 1 --2- 172.21.15.93:0/623719986 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c2c0ff160 0x7f8c2c10d110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.958+0000 7f8c31331700 1 --2- 172.21.15.93:0/623719986 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c2c0fe7b0 0x7f8c2c0fec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.958+0000 7f8c31331700 1 -- 172.21.15.93:0/623719986 >> 172.21.15.93:0/623719986 conn(0x7f8c2c075810 msgr2=0x7f8c2c075c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.958+0000 7f8c31331700 1 -- 172.21.15.93:0/623719986 shutdown_connections 2023-12-14T18:28:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.958+0000 7f8c31331700 1 -- 172.21.15.93:0/623719986 wait complete. 2023-12-14T18:28:41.964 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c31331700 1 Processor -- start 2023-12-14T18:28:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c31331700 1 -- start start 2023-12-14T18:28:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c31331700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c2c0fe7b0 0x7f8c2c196510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c31331700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c0ff160 0x7f8c2c196a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c31331700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c2c106350 0x7f8c2c19aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c31331700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c19b4f0 con 0x7f8c2c0ff160 2023-12-14T18:28:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c31331700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c19b660 con 0x7f8c2c106350 2023-12-14T18:28:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c31331700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c2c19b820 con 0x7f8c2c0fe7b0 2023-12-14T18:28:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c2a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c0ff160 0x7f8c2c196a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:41.965 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c2a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c0ff160 0x7f8c2c196a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46888/0 (socket says 172.21.15.93:46888) 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c2a7fc700 1 -- 172.21.15.93:0/3156373556 learned_addr learned my addr 172.21.15.93:0/3156373556 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c2c106350 0x7f8c2c19aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.959+0000 7f8c2affd700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c2c0fe7b0 0x7f8c2c196510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/3156373556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c2c0fe7b0 msgr2=0x7f8c2c196510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c2c0fe7b0 0x7f8c2c196510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c0ff160 msgr2=0x7f8c2c196a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c0ff160 0x7f8c2c196a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/3156373556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c1c0095f0 con 0x7f8c2c106350 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c2a7fc700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c0ff160 0x7f8c2c196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:41.966 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c2c106350 0x7f8c2c19aeb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400ab40 tx=0x7f8c2400ae50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c13fff700 1 -- 172.21.15.93:0/3156373556 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2400f070 con 0x7f8c2c106350 2023-12-14T18:28:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c13fff700 1 -- 172.21.15.93:0/3156373556 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c24007a70 con 0x7f8c2c106350 2023-12-14T18:28:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c31331700 1 -- 172.21.15.93:0/3156373556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c2c1103a0 con 0x7f8c2c106350 2023-12-14T18:28:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c13fff700 1 -- 172.21.15.93:0/3156373556 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24005280 con 0x7f8c2c106350 2023-12-14T18:28:41.967 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.960+0000 7f8c31331700 1 -- 172.21.15.93:0/3156373556 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c110210 con 0x7f8c2c106350 2023-12-14T18:28:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.962+0000 7f8c13fff700 1 -- 172.21.15.93:0/3156373556 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8c24004ad0 con 0x7f8c2c106350 2023-12-14T18:28:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.963+0000 7f8c13fff700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8c140a1340 0x7f8c140a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.963+0000 7f8c13fff700 1 -- 172.21.15.93:0/3156373556 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f8c240cc350 con 0x7f8c2c106350 2023-12-14T18:28:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.963+0000 7f8c2affd700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8c140a1340 0x7f8c140a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:41.968 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:41.963+0000 7f8c2affd700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8c140a1340 0x7f8c140a37f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00b2e0 tx=0x7f8c1c005fb0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c3d4bfb80 1 Processor -- start 2023-12-14T18:28:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c3d4bfb80 1 -- start start 2023-12-14T18:28:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 0x56135a1fee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a1ff370 0x56135a1ff7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a20c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a0e9580 con 0x56135a1ff370 2023-12-14T18:28:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a158810 con 0x56135a20a0f0 2023-12-14T18:28:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a151ca0 con 0x56135a05c430 2023-12-14T18:28:42.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c2a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a1ff370 0x56135a1ff7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c2a7fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a1ff370 0x56135a1ff7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46894/0 (socket says 172.21.15.93:46894) 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c2a7fc700 1 -- 172.21.15.93:0/2068312139 learned_addr learned my addr 172.21.15.93:0/2068312139 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.001+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a20c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c2affd700 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 0x56135a1fee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c2a7fc700 1 -- 172.21.15.93:0/2068312139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 msgr2=0x56135a1fee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c2a7fc700 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 0x56135a1fee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c2a7fc700 1 -- 172.21.15.93:0/2068312139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 msgr2=0x56135a20c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c2a7fc700 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a20c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c2a7fc700 1 -- 172.21.15.93:0/2068312139 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561359fd3350 con 0x56135a1ff370 2023-12-14T18:28:42.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c2affd700 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 0x56135a1fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a20c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c2a7fc700 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a1ff370 0x56135a1ff7e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8c20011680 tx=0x7f8c20011990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d102c2175421882d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c117fa700 1 -- 172.21.15.93:0/2068312139 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c20004030 con 0x56135a1ff370 2023-12-14T18:28:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c117fa700 1 -- 172.21.15.93:0/2068312139 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c20003680 con 0x56135a1ff370 2023-12-14T18:28:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.002+0000 7f8c117fa700 1 -- 172.21.15.93:0/2068312139 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c200196b0 con 0x56135a1ff370 2023-12-14T18:28:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.003+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2068312139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a1ff370 msgr2=0x56135a1ff7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.003+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a1ff370 0x56135a1ff7e0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f8c20011680 tx=0x7f8c20011990 comp rx=0 tx=0).stop 2023-12-14T18:28:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.003+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2068312139 shutdown_connections 2023-12-14T18:28:42.013 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.003+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a20c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.003+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 0x56135a1fee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.003+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/2068312139 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a1ff370 0x56135a1ff7e0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.003+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2068312139 >> 172.21.15.93:0/2068312139 conn(0x56135a1f5890 msgr2=0x56135a1fb200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.003+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2068312139 shutdown_connections 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.003+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2068312139 wait complete. 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c3d4bfb80 1 -- start start 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c3d4bfb80 1 Processor -- start 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a05c430 0x56135a209a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 0x56135a206d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a207350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.014 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a0edf30 con 0x56135a05c430 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a0edbd0 con 0x56135a20a0f0 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561359fd5640 con 0x56135a1ff370 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c3d4bfb80 10 client.0 _ll_register_callbacks cb 0x56135a1118b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c2affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a05c430 0x56135a209a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c2affd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a05c430 0x56135a209a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46900/0 (socket says 172.21.15.93:46900) 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c2b7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a207350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c2affd700 1 -- 172.21.15.93:0/1311079223 learned_addr learned my addr 172.21.15.93:0/1311079223 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c2a7fc700 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 0x56135a206d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.004+0000 7f8c2affd700 1 -- 172.21.15.93:0/1311079223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 msgr2=0x56135a206d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:42.015 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c2affd700 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 0x56135a206d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c2affd700 1 -- 172.21.15.93:0/1311079223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 msgr2=0x56135a207350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c2affd700 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a207350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c2affd700 1 -- 172.21.15.93:0/1311079223 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c20009e10 con 0x56135a05c430 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a207350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c2affd700 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a05c430 0x56135a209a50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c005770 tx=0x7f8c1c009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32005685a06009ad server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c0eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1311079223 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c1c00bee0 con 0x56135a05c430 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1311079223 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56135a1f2ea0 con 0x56135a05c430 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1311079223 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f8c1c00f660 con 0x56135a05c430 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1311079223 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c1c00f820 con 0x56135a05c430 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c3d4bfb80 10 client.15717 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:42.016 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1311079223 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56135a118110 con 0x56135a05c430 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.005+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1311079223 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56135a1f2ea0 con 0x56135a05c430 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1311079223 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c1c031ac0 con 0x56135a05c430 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c3d4bfb80 10 client.15717 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c3d4bfb80 10 client.15717 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c3d4bfb80 20 client.15717 populate_metadata read hostname 'smithi093' 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56135a219c30 0x56135a21c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c3d4bfb80 4 client.15717 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1311079223 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56135a14ac80 con 0x56135a219c30 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c2a7fc700 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56135a219c30 0x56135a21c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.017 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c2a7fc700 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56135a219c30 0x56135a21c040 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=160f2dee24e46913 server_cookie=b276cf6e628045b1 in_seq=0 out_seq=0 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.006+0000 7f8c0eff5700 10 client.15717 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1311079223 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56135a14ac80 con 0x56135a219c30 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c0eff5700 10 client.15717 handle_command_reply: tid=0 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 client.15717 shutdown 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1311079223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a05c430 msgr2=0x56135a209a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a05c430 0x56135a209a50 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c005770 tx=0x7f8c1c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1311079223 shutdown_connections 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56135a219c30 0x56135a21c040 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a05c430 0x56135a209a50 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a20a0f0 0x56135a207350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1311079223 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 0x56135a206d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1311079223 >> 172.21.15.93:0/1311079223 conn(0x56135a1f5890 msgr2=0x56135a1fadf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c0eff5700 0 client.15717 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1311079223 shutdown_connections 2023-12-14T18:28:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1311079223 wait complete. 2023-12-14T18:28:42.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.008+0000 7f8c3d4bfb80 20 client.15717 trim_cache size 0 max 16384 2023-12-14T18:28:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c3d4bfb80 1 Processor -- start 2023-12-14T18:28:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c3d4bfb80 1 -- start start 2023-12-14T18:28:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 0x56135a210cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a1ff370 0x56135a21bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a202020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a208350 con 0x56135a20a0f0 2023-12-14T18:28:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a1fc7d0 con 0x56135a1ff370 2023-12-14T18:28:42.035 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a1ff740 con 0x56135a05c430 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a202020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a202020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46904/0 (socket says 172.21.15.93:46904) 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/2067003481 learned_addr learned my addr 172.21.15.93:0/2067003481 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c2a7fc700 1 --2- 172.21.15.93:0/2067003481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a1ff370 0x56135a21bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.027+0000 7f8c2affd700 1 --2- 172.21.15.93:0/2067003481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 0x56135a210cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.028+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/2067003481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 msgr2=0x56135a210cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.028+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/2067003481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 0x56135a210cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.028+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/2067003481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a1ff370 msgr2=0x56135a21bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.028+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/2067003481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a1ff370 0x56135a21bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.028+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/2067003481 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56135a117e60 con 0x56135a20a0f0 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.028+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/2067003481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a202020 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8c24095c30 tx=0x7f8c24095500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc37ced1f6fb138d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:42.036 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.028+0000 7f8c11ffb700 1 -- 172.21.15.93:0/2067003481 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c2409d1f0 con 0x56135a20a0f0 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.028+0000 7f8c11ffb700 1 -- 172.21.15.93:0/2067003481 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c240a1580 con 0x56135a20a0f0 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.028+0000 7f8c11ffb700 1 -- 172.21.15.93:0/2067003481 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24098910 con 0x56135a20a0f0 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2067003481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 msgr2=0x56135a202020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/2067003481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a202020 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f8c24095c30 tx=0x7f8c24095500 comp rx=0 tx=0).stop 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2067003481 shutdown_connections 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/2067003481 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a202020 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/2067003481 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a05c430 0x56135a210cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/2067003481 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a1ff370 0x56135a21bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2067003481 >> 172.21.15.93:0/2067003481 conn(0x56135a1f5890 msgr2=0x56135a215460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2067003481 shutdown_connections 2023-12-14T18:28:42.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/2067003481 wait complete. 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 -- start start 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 Processor -- start 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a05c430 0x56135a20eca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 0x56135a20f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a20f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a26caf0 con 0x56135a20a0f0 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a26cc30 con 0x56135a05c430 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.029+0000 7f8c3d4bfb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56135a26cdc0 con 0x56135a1ff370 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c3d4bfb80 10 client.0 _ll_register_callbacks cb 0x7f8c1c005350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a20f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a05c430 0x56135a20eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a20f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:46912/0 (socket says 172.21.15.93:46912) 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2affd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a05c430 0x56135a20eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:51606/0 (socket says 172.21.15.93:51606) 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/1419026565 learned_addr learned my addr 172.21.15.93:0/1419026565 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2a7fc700 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 0x56135a20f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/1419026565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 msgr2=0x56135a20f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 0x56135a20f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/1419026565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a05c430 msgr2=0x56135a20eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a05c430 0x56135a20eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2b7fe700 1 -- 172.21.15.93:0/1419026565 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c2c1103a0 con 0x56135a20a0f0 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2affd700 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a05c430 0x56135a20eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c2b7fe700 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a20f720 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c2409f210 tx=0x7f8c24098a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7403d24efdbaf715 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:42.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c0eff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1419026565 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c24004dd0 con 0x56135a20a0f0 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1419026565 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8c20003680 con 0x56135a20a0f0 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.030+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1419026565 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f8c2409d1f0 con 0x56135a20a0f0 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1419026565 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8c24099630 con 0x56135a20a0f0 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c3d4bfb80 10 client.15723 fetch_fsmap learned FSMap version 24 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1419026565 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=4+}) v3 -- 0x56135a0fe940 con 0x56135a20a0f0 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1419026565 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8c20003680 con 0x56135a20a0f0 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1419026565 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 24) v1 ==== 7470+0+0 (secure 0 0 0) 0x7f8c24094fb0 con 0x56135a20a0f0 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c3d4bfb80 10 client.15723 fetch_fsmap finished waiting for FSMap version 24 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c3d4bfb80 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:28:42.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c3d4bfb80 20 client.15723 populate_metadata read hostname 'smithi093' 2023-12-14T18:28:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56135a26dff0 0x56135a2703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c3d4bfb80 4 client.15723 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:28:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1419026565 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56135a14a9a0 con 0x56135a26dff0 2023-12-14T18:28:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c2affd700 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56135a26dff0 0x56135a2703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.031+0000 7f8c0eff5700 10 client.15723 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:42.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.032+0000 7f8c2affd700 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56135a26dff0 0x56135a2703d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=589c0fedf944c649 server_cookie=5bc9787d20e67187 in_seq=0 out_seq=0 2023-12-14T18:28:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c0eff5700 1 -- 172.21.15.93:0/1419026565 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56135a14a9a0 con 0x56135a26dff0 2023-12-14T18:28:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c0eff5700 10 client.15723 handle_command_reply: tid=0 2023-12-14T18:28:42.931 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 client.15723 shutdown 2023-12-14T18:28:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1419026565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 msgr2=0x56135a20f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a20f720 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f8c2409f210 tx=0x7f8c24098a70 comp rx=0 tx=0).stop 2023-12-14T18:28:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1419026565 shutdown_connections 2023-12-14T18:28:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56135a26dff0 0x56135a2703d0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56135a20a0f0 0x56135a20f720 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56135a05c430 0x56135a20eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 --2- 172.21.15.93:0/1419026565 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56135a1ff370 0x56135a20f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1419026565 >> 172.21.15.93:0/1419026565 conn(0x56135a1f5890 msgr2=0x56135a202020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:42.932 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c0eff5700 0 client.15723 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1419026565 shutdown_connections 2023-12-14T18:28:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 1 -- 172.21.15.93:0/1419026565 wait complete. 2023-12-14T18:28:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.928+0000 7f8c3d4bfb80 20 client.15723 trim_cache size 0 max 16384 2023-12-14T18:28:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:28:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:28:42.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:28:42.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:28:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 -- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8c140a1340 msgr2=0x7f8c140a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8c140a1340 0x7f8c140a37f0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8c1c00b2e0 tx=0x7f8c1c005fb0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 -- 172.21.15.93:0/3156373556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c2c106350 msgr2=0x7f8c2c19aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c2c106350 0x7f8c2c19aeb0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8c2400ab40 tx=0x7f8c2400ae50 comp rx=0 tx=0).stop 2023-12-14T18:28:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 -- 172.21.15.93:0/3156373556 shutdown_connections 2023-12-14T18:28:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8c2c106350 0x7f8c2c19aeb0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8c2c0fe7b0 0x7f8c2c196510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8c140a1340 0x7f8c140a37f0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 --2- 172.21.15.93:0/3156373556 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8c2c0ff160 0x7f8c2c196a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 -- 172.21.15.93:0/3156373556 >> 172.21.15.93:0/3156373556 conn(0x7f8c2c075810 msgr2=0x7f8c2c0fc280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 -- 172.21.15.93:0/3156373556 shutdown_connections 2023-12-14T18:28:42.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:42.929+0000 7f8c31331700 1 -- 172.21.15.93:0/3156373556 wait complete. 2023-12-14T18:28:43.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:42 smithi160 ceph-mon[43840]: pgmap v70: 129 pgs: 129 active+clean; 2.9 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-12-14T18:28:43.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:42 smithi093 ceph-mon[42070]: pgmap v70: 129 pgs: 129 active+clean; 2.9 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-12-14T18:28:43.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:42 smithi097 ceph-mon[43486]: pgmap v70: 129 pgs: 129 active+clean; 2.9 MiB data, 87 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 7 op/s 2023-12-14T18:28:45.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:44 smithi160 ceph-mon[43840]: pgmap v71: 129 pgs: 129 active+clean; 5.2 MiB data, 118 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 686 KiB/s wr, 497 op/s 2023-12-14T18:28:45.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:44 smithi093 ceph-mon[42070]: pgmap v71: 129 pgs: 129 active+clean; 5.2 MiB data, 118 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 686 KiB/s wr, 497 op/s 2023-12-14T18:28:45.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:44 smithi097 ceph-mon[43486]: pgmap v71: 129 pgs: 129 active+clean; 5.2 MiB data, 118 MiB used, 1.0 TiB / 1.0 TiB avail; 557 KiB/s rd, 686 KiB/s wr, 497 op/s 2023-12-14T18:28:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:46 smithi160 ceph-mon[43840]: pgmap v72: 129 pgs: 129 active+clean; 5.2 MiB data, 118 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 682 KiB/s wr, 494 op/s 2023-12-14T18:28:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:46 smithi093 ceph-mon[42070]: pgmap v72: 129 pgs: 129 active+clean; 5.2 MiB data, 118 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 682 KiB/s wr, 494 op/s 2023-12-14T18:28:47.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:46 smithi097 ceph-mon[43486]: pgmap v72: 129 pgs: 129 active+clean; 5.2 MiB data, 118 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 682 KiB/s wr, 494 op/s 2023-12-14T18:28:49.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:48 smithi160 ceph-mon[43840]: pgmap v73: 129 pgs: 129 active+clean; 5.2 MiB data, 118 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 682 KiB/s wr, 494 op/s 2023-12-14T18:28:49.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:48 smithi093 ceph-mon[42070]: pgmap v73: 129 pgs: 129 active+clean; 5.2 MiB data, 118 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 682 KiB/s wr, 494 op/s 2023-12-14T18:28:49.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:48 smithi097 ceph-mon[43486]: pgmap v73: 129 pgs: 129 active+clean; 5.2 MiB data, 118 MiB used, 1.0 TiB / 1.0 TiB avail; 546 KiB/s rd, 682 KiB/s wr, 494 op/s 2023-12-14T18:28:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:50 smithi160 ceph-mon[43840]: pgmap v74: 129 pgs: 129 active+clean; 17 MiB data, 143 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 566 op/s 2023-12-14T18:28:51.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:50 smithi093 ceph-mon[42070]: pgmap v74: 129 pgs: 129 active+clean; 17 MiB data, 143 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 566 op/s 2023-12-14T18:28:51.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:50 smithi097 ceph-mon[43486]: pgmap v74: 129 pgs: 129 active+clean; 17 MiB data, 143 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 566 op/s 2023-12-14T18:28:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:52 smithi160 ceph-mon[43840]: pgmap v75: 129 pgs: 129 active+clean; 17 MiB data, 143 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 562 op/s 2023-12-14T18:28:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:52 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:28:53.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:52 smithi093 ceph-mon[42070]: pgmap v75: 129 pgs: 129 active+clean; 17 MiB data, 143 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 562 op/s 2023-12-14T18:28:53.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:52 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:28:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:52 smithi097 ceph-mon[43486]: pgmap v75: 129 pgs: 129 active+clean; 17 MiB data, 143 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.0 MiB/s wr, 562 op/s 2023-12-14T18:28:53.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:52 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:28:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:54 smithi160 ceph-mon[43840]: pgmap v76: 129 pgs: 129 active+clean; 25 MiB data, 158 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 568 op/s 2023-12-14T18:28:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:54 smithi093 ceph-mon[42070]: pgmap v76: 129 pgs: 129 active+clean; 25 MiB data, 158 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 568 op/s 2023-12-14T18:28:55.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:54 smithi097 ceph-mon[43486]: pgmap v76: 129 pgs: 129 active+clean; 25 MiB data, 158 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.7 MiB/s wr, 568 op/s 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.052+0000 7f30e7d0f700 1 -- 172.21.15.93:0/128640009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e00ffd60 msgr2=0x7f30e0103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.052+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/128640009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e00ffd60 0x7f30e0103d20 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f30dc0099c0 tx=0x7f30dc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.052+0000 7f30e7d0f700 1 -- 172.21.15.93:0/128640009 shutdown_connections 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.052+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/128640009 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e0104400 0x7f30e010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.052+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/128640009 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e00ff3c0 0x7f30e00ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.052+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/128640009 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e00ffd60 0x7f30e0103d20 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.052+0000 7f30e7d0f700 1 -- 172.21.15.93:0/128640009 >> 172.21.15.93:0/128640009 conn(0x7f30e00763a0 msgr2=0x7f30e00767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 -- 172.21.15.93:0/128640009 shutdown_connections 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 -- 172.21.15.93:0/128640009 wait complete. 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 Processor -- start 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 -- start start 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e00ff3c0 0x7f30e01952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e00ffd60 0x7f30e01957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.056 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e0104400 0x7f30e0199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e019a280 con 0x7f30e0104400 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e019a3f0 con 0x7f30e00ffd60 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.053+0000 7f30e7d0f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30e019a5b0 con 0x7f30e00ff3c0 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e62ac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e0104400 0x7f30e0199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e62ac700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e0104400 0x7f30e0199c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43446/0 (socket says 172.21.15.93:43446) 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e62ac700 1 -- 172.21.15.93:0/1437934358 learned_addr learned my addr 172.21.15.93:0/1437934358 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e52aa700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e00ffd60 0x7f30e01957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e5aab700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e00ff3c0 0x7f30e01952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e62ac700 1 -- 172.21.15.93:0/1437934358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e00ff3c0 msgr2=0x7f30e01952a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e62ac700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e00ff3c0 0x7f30e01952a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e62ac700 1 -- 172.21.15.93:0/1437934358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e00ffd60 msgr2=0x7f30e01957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e62ac700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e00ffd60 0x7f30e01957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.057 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e62ac700 1 -- 172.21.15.93:0/1437934358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30dc0095f0 con 0x7f30e0104400 2023-12-14T18:28:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e5aab700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e00ff3c0 0x7f30e01952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:28:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e62ac700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e0104400 0x7f30e0199c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f30d8007ee0 tx=0x7f30d800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30d6ffd700 1 -- 172.21.15.93:0/1437934358 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8017070 con 0x7f30e0104400 2023-12-14T18:28:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e7d0f700 1 -- 172.21.15.93:0/1437934358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30e010f180 con 0x7f30e0104400 2023-12-14T18:28:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30d6ffd700 1 -- 172.21.15.93:0/1437934358 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30d800f040 con 0x7f30e0104400 2023-12-14T18:28:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30d6ffd700 1 -- 172.21.15.93:0/1437934358 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d8005740 con 0x7f30e0104400 2023-12-14T18:28:56.058 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.054+0000 7f30e7d0f700 1 -- 172.21.15.93:0/1437934358 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e010eff0 con 0x7f30e0104400 2023-12-14T18:28:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.057+0000 7f30d6ffd700 1 -- 172.21.15.93:0/1437934358 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f30d80058a0 con 0x7f30e0104400 2023-12-14T18:28:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.057+0000 7f30d6ffd700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f30cc0a12c0 0x7f30cc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.064 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.057+0000 7f30d6ffd700 1 -- 172.21.15.93:0/1437934358 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f30d80cc830 con 0x7f30e0104400 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.057+0000 7f30e5aab700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f30cc0a12c0 0x7f30cc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.058+0000 7f30e5aab700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f30cc0a12c0 0x7f30cc0a3770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30d0005fd0 tx=0x7f30d0005dc0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.058+0000 7f30f3e9db80 1 Processor -- start 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.058+0000 7f30f3e9db80 1 -- start start 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f43e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f447e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f514d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32e2e580 con 0x55ef32f44370 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32e9d810 con 0x55ef32f4f0f0 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32e96ca0 con 0x55ef32da1430 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e52aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f447e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.065 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e52aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f447e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43456/0 (socket says 172.21.15.93:43456) 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e52aa700 1 -- 172.21.15.93:0/3251719760 learned_addr learned my addr 172.21.15.93:0/3251719760 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e5aab700 1 --2- 172.21.15.93:0/3251719760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f43e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e62ac700 1 --2- 172.21.15.93:0/3251719760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f514d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e52aa700 1 -- 172.21.15.93:0/3251719760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 msgr2=0x55ef32f43e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e52aa700 1 --2- 172.21.15.93:0/3251719760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f43e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e52aa700 1 -- 172.21.15.93:0/3251719760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 msgr2=0x55ef32f514d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e52aa700 1 --2- 172.21.15.93:0/3251719760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f514d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e52aa700 1 -- 172.21.15.93:0/3251719760 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef32d18350 con 0x55ef32f44370 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30e52aa700 1 --2- 172.21.15.93:0/3251719760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f447e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f30e0100130 tx=0x7f30dc005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c05f10dc449695 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.059+0000 7f30cbfff700 1 -- 172.21.15.93:0/3251719760 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc0052c0 con 0x55ef32f44370 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30cbfff700 1 -- 172.21.15.93:0/3251719760 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30dc01e040 con 0x55ef32f44370 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30cbfff700 1 -- 172.21.15.93:0/3251719760 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc003b20 con 0x55ef32f44370 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30f3e9db80 1 -- 172.21.15.93:0/3251719760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 msgr2=0x55ef32f447e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/3251719760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f447e0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f30e0100130 tx=0x7f30dc005c70 comp rx=0 tx=0).stop 2023-12-14T18:28:56.066 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30f3e9db80 1 -- 172.21.15.93:0/3251719760 shutdown_connections 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/3251719760 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f514d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/3251719760 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f43e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/3251719760 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f447e0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30f3e9db80 1 -- 172.21.15.93:0/3251719760 >> 172.21.15.93:0/3251719760 conn(0x55ef32f3a890 msgr2=0x55ef32f40200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30f3e9db80 1 -- 172.21.15.93:0/3251719760 shutdown_connections 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.060+0000 7f30f3e9db80 1 -- 172.21.15.93:0/3251719760 wait complete. 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30f3e9db80 1 -- start start 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30f3e9db80 1 Processor -- start 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f4eac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f4bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f4c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32e32f30 con 0x55ef32f44370 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32e32bd0 con 0x55ef32f4f0f0 2023-12-14T18:28:56.067 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32d1a640 con 0x55ef32da1430 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30e52aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f4bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30f3e9db80 10 client.0 _ll_register_callbacks cb 0x55ef32e568b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30e52aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f4bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43460/0 (socket says 172.21.15.93:43460) 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30e52aa700 1 -- 172.21.15.93:0/1403421086 learned_addr learned my addr 172.21.15.93:0/1403421086 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.061+0000 7f30e5aab700 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f4eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30e52aa700 1 -- 172.21.15.93:0/1403421086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 msgr2=0x55ef32f4eac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30e52aa700 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f4eac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30e52aa700 1 -- 172.21.15.93:0/1403421086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 msgr2=0x55ef32f4c460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30e52aa700 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f4c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30e52aa700 1 -- 172.21.15.93:0/1403421086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30e010c810 con 0x55ef32f44370 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30e52aa700 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f4bde0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f30dc005920 tx=0x7f30dc005ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dff67a4eddd93e0e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30c97fa700 1 -- 172.21.15.93:0/1403421086 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc00bd60 con 0x55ef32f44370 2023-12-14T18:28:56.068 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30c97fa700 1 -- 172.21.15.93:0/1403421086 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f30dc0038c0 con 0x55ef32f44370 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30f3e9db80 1 -- 172.21.15.93:0/1403421086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ef32f37ea0 con 0x55ef32f44370 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30c97fa700 1 -- 172.21.15.93:0/1403421086 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30dc00bec0 con 0x55ef32f44370 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30f3e9db80 10 client.15735 fetch_fsmap learned FSMap version 25 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30f3e9db80 1 -- 172.21.15.93:0/1403421086 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ef32e5d110 con 0x55ef32f44370 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30c97fa700 1 -- 172.21.15.93:0/1403421086 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ef32f37ea0 con 0x55ef32f44370 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30c97fa700 1 -- 172.21.15.93:0/1403421086 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f30dc004950 con 0x55ef32f44370 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.062+0000 7f30f3e9db80 10 client.15735 fetch_fsmap finished waiting for FSMap version 25 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.063+0000 7f30f3e9db80 10 client.15735 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.063+0000 7f30f3e9db80 20 client.15735 populate_metadata read hostname 'smithi093' 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.063+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ef32f5e740 0x55ef32f60b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.063+0000 7f30f3e9db80 4 client.15735 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.063+0000 7f30f3e9db80 1 -- 172.21.15.93:0/1403421086 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ef32e8fc80 con 0x55ef32f5e740 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.063+0000 7f30e5aab700 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ef32f5e740 0x55ef32f60b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.063+0000 7f30c97fa700 10 client.15735 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:56.069 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.063+0000 7f30e5aab700 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ef32f5e740 0x55ef32f60b50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b68eb9b1ce611d0 server_cookie=233c1e6f72f44086 in_seq=0 out_seq=0 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30c97fa700 1 -- 172.21.15.93:0/1403421086 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ef32e8fc80 con 0x55ef32f5e740 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30c97fa700 10 client.15735 handle_command_reply: tid=0 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 client.15735 shutdown 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 -- 172.21.15.93:0/1403421086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 msgr2=0x55ef32f4bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f4bde0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f30dc005920 tx=0x7f30dc005ae0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 -- 172.21.15.93:0/1403421086 shutdown_connections 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ef32f5e740 0x55ef32f60b50 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f4c460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f4eac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30c97fa700 0 client.15735 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/1403421086 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f4bde0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 -- 172.21.15.93:0/1403421086 >> 172.21.15.93:0/1403421086 conn(0x55ef32f3a890 msgr2=0x55ef32f3fdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.071 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 -- 172.21.15.93:0/1403421086 shutdown_connections 2023-12-14T18:28:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 1 -- 172.21.15.93:0/1403421086 wait complete. 2023-12-14T18:28:56.072 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.064+0000 7f30f3e9db80 20 client.15735 trim_cache size 0 max 16384 2023-12-14T18:28:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30f3e9db80 1 Processor -- start 2023-12-14T18:28:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30f3e9db80 1 -- start start 2023-12-14T18:28:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f55e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f3f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f4e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32f3c9b0 con 0x55ef32f44370 2023-12-14T18:28:56.085 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32f450a0 con 0x55ef32f4f0f0 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32f45d30 con 0x55ef32da1430 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30e52aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f3f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30e52aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f3f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43468/0 (socket says 172.21.15.93:43468) 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.079+0000 7f30e52aa700 1 -- 172.21.15.93:0/2757066052 learned_addr learned my addr 172.21.15.93:0/2757066052 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30e5aab700 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f55e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30e52aa700 1 -- 172.21.15.93:0/2757066052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 msgr2=0x55ef32f55e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30e52aa700 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f55e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30e52aa700 1 -- 172.21.15.93:0/2757066052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 msgr2=0x55ef32f4e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30e62ac700 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f4e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30e52aa700 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f4e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30e52aa700 1 -- 172.21.15.93:0/2757066052 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ef32e5ce60 con 0x55ef32f44370 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30e62ac700 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f4e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30e52aa700 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f3f3d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f30dc02c460 tx=0x7f30dc009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bbd02b93d0a216a server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30d4ff9700 1 -- 172.21.15.93:0/2757066052 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc004db0 con 0x55ef32f44370 2023-12-14T18:28:56.086 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30d4ff9700 1 -- 172.21.15.93:0/2757066052 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30dc004500 con 0x55ef32f44370 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.080+0000 7f30d4ff9700 1 -- 172.21.15.93:0/2757066052 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30dc003b20 con 0x55ef32f44370 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 -- 172.21.15.93:0/2757066052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 msgr2=0x55ef32f3f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f3f3d0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f30dc02c460 tx=0x7f30dc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 -- 172.21.15.93:0/2757066052 shutdown_connections 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32f4e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32da1430 0x55ef32f55e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/2757066052 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32f44370 0x55ef32f3f3d0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 -- 172.21.15.93:0/2757066052 >> 172.21.15.93:0/2757066052 conn(0x55ef32f3a890 msgr2=0x55ef32f47390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 -- 172.21.15.93:0/2757066052 shutdown_connections 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 -- 172.21.15.93:0/2757066052 wait complete. 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 -- start start 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 Processor -- start 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32da1430 0x55ef32fa8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32f44370 0x55ef32fa8c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.081+0000 7f30f3e9db80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32fb35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.087 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32fb3b20 con 0x55ef32da1430 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32fb3c90 con 0x55ef32f4f0f0 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30f3e9db80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ef32fb3e50 con 0x55ef32f44370 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30f3e9db80 10 client.0 _ll_register_callbacks cb 0x7f30d8095950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e5aab700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32da1430 0x55ef32fa8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e5aab700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32da1430 0x55ef32fa8750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43472/0 (socket says 172.21.15.93:43472) 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e5aab700 1 -- 172.21.15.93:0/25900272 learned_addr learned my addr 172.21.15.93:0/25900272 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e52aa700 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32f44370 0x55ef32fa8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e62ac700 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32fb35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e5aab700 1 -- 172.21.15.93:0/25900272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32f44370 msgr2=0x55ef32fa8c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e5aab700 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32f44370 0x55ef32fa8c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e5aab700 1 -- 172.21.15.93:0/25900272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 msgr2=0x55ef32fb35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e5aab700 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32fb35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:56.088 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e5aab700 1 -- 172.21.15.93:0/25900272 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ef32e5d110 con 0x55ef32da1430 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e62ac700 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32fb35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e52aa700 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32f44370 0x55ef32fa8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30e5aab700 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32da1430 0x55ef32fa8750 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55ef32f613f0 tx=0x7f30d0004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16510abe6becef1c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30c97fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30c97fa700 1 -- 172.21.15.93:0/25900272 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30d0022040 con 0x55ef32da1430 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30c97fa700 1 -- 172.21.15.93:0/25900272 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f30d001d430 con 0x55ef32da1430 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30f3e9db80 1 -- 172.21.15.93:0/25900272 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f30dc01e040 con 0x55ef32da1430 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30c97fa700 1 -- 172.21.15.93:0/25900272 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=25) v2 ==== 24+0+0 (secure 0 0 0) 0x7f30d0028e60 con 0x55ef32da1430 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30f3e9db80 10 client.15741 fetch_fsmap learned FSMap version 25 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.082+0000 7f30f3e9db80 1 -- 172.21.15.93:0/25900272 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=25,monmap=4+}) v3 -- 0x55ef32e43940 con 0x55ef32da1430 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30c97fa700 1 -- 172.21.15.93:0/25900272 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f30dc01e040 con 0x55ef32da1430 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30c97fa700 1 -- 172.21.15.93:0/25900272 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 25) v1 ==== 7474+0+0 (secure 0 0 0) 0x7f30d001d950 con 0x55ef32da1430 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30f3e9db80 10 client.15741 fetch_fsmap finished waiting for FSMap version 25 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30f3e9db80 10 client.15741 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:28:56.089 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30f3e9db80 20 client.15741 populate_metadata read hostname 'smithi093' 2023-12-14T18:28:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ef32fb55e0 0x55ef32fb79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:28:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30f3e9db80 4 client.15741 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:28:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30f3e9db80 1 -- 172.21.15.93:0/25900272 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ef32e8f9a0 con 0x55ef32fb55e0 2023-12-14T18:28:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30e52aa700 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ef32fb55e0 0x55ef32fb79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:28:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30c97fa700 10 client.15741 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:56.090 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:56.083+0000 7f30e52aa700 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ef32fb55e0 0x55ef32fb79f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8bd68ac7350a4d81 server_cookie=1f18b34b20e3d7e0 in_seq=0 out_seq=0 2023-12-14T18:28:57.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:56 smithi160 ceph-mon[43840]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 158 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 78 op/s 2023-12-14T18:28:57.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:56 smithi093 ceph-mon[42070]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 158 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 78 op/s 2023-12-14T18:28:57.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:56 smithi097 ceph-mon[43486]: pgmap v77: 129 pgs: 129 active+clean; 25 MiB data, 158 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 78 op/s 2023-12-14T18:28:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30c97fa700 1 -- 172.21.15.93:0/25900272 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ef32e8f9a0 con 0x55ef32fb55e0 2023-12-14T18:28:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30c97fa700 10 client.15741 handle_command_reply: tid=0 2023-12-14T18:28:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 client.15741 shutdown 2023-12-14T18:28:57.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 -- 172.21.15.93:0/25900272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32da1430 msgr2=0x55ef32fa8750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32da1430 0x55ef32fa8750 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x55ef32f613f0 tx=0x7f30d0004750 comp rx=0 tx=0).stop 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 -- 172.21.15.93:0/25900272 shutdown_connections 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ef32fb55e0 0x55ef32fb79f0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ef32da1430 0x55ef32fa8750 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ef32f4f0f0 0x55ef32fb35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 --2- 172.21.15.93:0/25900272 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ef32f44370 0x55ef32fa8c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 -- 172.21.15.93:0/25900272 >> 172.21.15.93:0/25900272 conn(0x55ef32f3a890 msgr2=0x55ef32f3f3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30c97fa700 0 client.15741 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.772+0000 7f30f3e9db80 1 -- 172.21.15.93:0/25900272 shutdown_connections 2023-12-14T18:28:57.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30f3e9db80 1 -- 172.21.15.93:0/25900272 wait complete. 2023-12-14T18:28:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30f3e9db80 20 client.15741 trim_cache size 0 max 16384 2023-12-14T18:28:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:28:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:28:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:28:57.776 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:28:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 -- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f30cc0a12c0 msgr2=0x7f30cc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:57.777 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f30cc0a12c0 0x7f30cc0a3770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f30d0005fd0 tx=0x7f30d0005dc0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 -- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e0104400 msgr2=0x7f30e0199c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:28:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e0104400 0x7f30e0199c40 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f30d8007ee0 tx=0x7f30d800d3b0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 -- 172.21.15.93:0/1437934358 shutdown_connections 2023-12-14T18:28:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f30cc0a12c0 0x7f30cc0a3770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f30e0104400 0x7f30e0199c40 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f30e00ff3c0 0x7f30e01952a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 --2- 172.21.15.93:0/1437934358 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f30e00ffd60 0x7f30e01957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:28:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 -- 172.21.15.93:0/1437934358 >> 172.21.15.93:0/1437934358 conn(0x7f30e00763a0 msgr2=0x7f30e0102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:28:57.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 -- 172.21.15.93:0/1437934358 shutdown_connections 2023-12-14T18:28:57.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:28:57.773+0000 7f30e7d0f700 1 -- 172.21.15.93:0/1437934358 wait complete. 2023-12-14T18:28:59.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:28:58 smithi160 ceph-mon[43840]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 158 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 78 op/s 2023-12-14T18:28:59.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:28:58 smithi093 ceph-mon[42070]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 158 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 78 op/s 2023-12-14T18:28:59.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:28:58 smithi097 ceph-mon[43486]: pgmap v78: 129 pgs: 129 active+clean; 25 MiB data, 158 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.0 MiB/s wr, 78 op/s 2023-12-14T18:29:01.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:00 smithi160 ceph-mon[43840]: pgmap v79: 129 pgs: 129 active+clean; 9.9 MiB data, 186 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 518 op/s 2023-12-14T18:29:01.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:00 smithi093 ceph-mon[42070]: pgmap v79: 129 pgs: 129 active+clean; 9.9 MiB data, 186 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 518 op/s 2023-12-14T18:29:01.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:00 smithi097 ceph-mon[43486]: pgmap v79: 129 pgs: 129 active+clean; 9.9 MiB data, 186 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.8 MiB/s wr, 518 op/s 2023-12-14T18:29:03.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:02 smithi160 ceph-mon[43840]: pgmap v80: 129 pgs: 129 active+clean; 9.9 MiB data, 186 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 446 op/s 2023-12-14T18:29:03.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:02 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:29:03.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:02 smithi093 ceph-mon[42070]: pgmap v80: 129 pgs: 129 active+clean; 9.9 MiB data, 186 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 446 op/s 2023-12-14T18:29:03.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:02 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:29:03.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:02 smithi097 ceph-mon[43486]: pgmap v80: 129 pgs: 129 active+clean; 9.9 MiB data, 186 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 MiB/s rd, 1.5 MiB/s wr, 446 op/s 2023-12-14T18:29:03.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:02 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:29:05.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:04 smithi160 ceph-mon[43840]: pgmap v81: 129 pgs: 129 active+clean; 33 MiB data, 204 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 465 op/s 2023-12-14T18:29:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:04 smithi093 ceph-mon[42070]: pgmap v81: 129 pgs: 129 active+clean; 33 MiB data, 204 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 465 op/s 2023-12-14T18:29:05.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:04 smithi097 ceph-mon[43486]: pgmap v81: 129 pgs: 129 active+clean; 33 MiB data, 204 MiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.5 MiB/s wr, 465 op/s 2023-12-14T18:29:07.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:06 smithi160 ceph-mon[43840]: pgmap v82: 129 pgs: 129 active+clean; 33 MiB data, 204 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 459 op/s 2023-12-14T18:29:07.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:06 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:29:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:06 smithi093 ceph-mon[42070]: pgmap v82: 129 pgs: 129 active+clean; 33 MiB data, 204 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 459 op/s 2023-12-14T18:29:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:06 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:29:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:06 smithi097 ceph-mon[43486]: pgmap v82: 129 pgs: 129 active+clean; 33 MiB data, 204 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 459 op/s 2023-12-14T18:29:07.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:06 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:29:09.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:08 smithi160 ceph-mon[43840]: pgmap v83: 129 pgs: 129 active+clean; 33 MiB data, 204 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 459 op/s 2023-12-14T18:29:09.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:08 smithi093 ceph-mon[42070]: pgmap v83: 129 pgs: 129 active+clean; 33 MiB data, 204 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 459 op/s 2023-12-14T18:29:09.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:08 smithi097 ceph-mon[43486]: pgmap v83: 129 pgs: 129 active+clean; 33 MiB data, 204 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 459 op/s 2023-12-14T18:29:10.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.898+0000 7f595abe8700 1 -- 172.21.15.93:0/1305175418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 msgr2=0x7f5954100890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.898+0000 7f595abe8700 1 --2- 172.21.15.93:0/1305175418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 0x7f5954100890 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f59480099c0 tx=0x7f5948009cd0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.898+0000 7f595abe8700 1 -- 172.21.15.93:0/1305175418 shutdown_connections 2023-12-14T18:29:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.898+0000 7f595abe8700 1 --2- 172.21.15.93:0/1305175418 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 0x7f5954100890 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.898+0000 7f595abe8700 1 --2- 172.21.15.93:0/1305175418 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5954105500 0x7f595410d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.898+0000 7f595abe8700 1 --2- 172.21.15.93:0/1305175418 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954100e60 0x7f5954104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.898+0000 7f595abe8700 1 -- 172.21.15.93:0/1305175418 >> 172.21.15.93:0/1305175418 conn(0x7f59540fbd50 msgr2=0x7f59540fe160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:10.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.898+0000 7f595abe8700 1 -- 172.21.15.93:0/1305175418 shutdown_connections 2023-12-14T18:29:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.898+0000 7f595abe8700 1 -- 172.21.15.93:0/1305175418 wait complete. 2023-12-14T18:29:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f595abe8700 1 Processor -- start 2023-12-14T18:29:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f595abe8700 1 -- start start 2023-12-14T18:29:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f595abe8700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 0x7f5954072df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f595abe8700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954100e60 0x7f5954073330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f595abe8700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5954105500 0x7f59541a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f595abe8700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5954073b60 con 0x7f59541004c0 2023-12-14T18:29:10.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f595abe8700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541a4af0 con 0x7f5954100e60 2023-12-14T18:29:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f595abe8700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59541a4c30 con 0x7f5954105500 2023-12-14T18:29:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f5958984700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 0x7f5954072df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f5958984700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 0x7f5954072df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55452/0 (socket says 172.21.15.93:55452) 2023-12-14T18:29:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.899+0000 7f5958984700 1 -- 172.21.15.93:0/2087592927 learned_addr learned my addr 172.21.15.93:0/2087592927 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f5953fff700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954100e60 0x7f5954073330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f5959185700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5954105500 0x7f59541a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f5958984700 1 -- 172.21.15.93:0/2087592927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5954105500 msgr2=0x7f59541a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f5958984700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5954105500 0x7f59541a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f5958984700 1 -- 172.21.15.93:0/2087592927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954100e60 msgr2=0x7f5954073330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f5958984700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954100e60 0x7f5954073330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f5958984700 1 -- 172.21.15.93:0/2087592927 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59480095f0 con 0x7f59541004c0 2023-12-14T18:29:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f5958984700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 0x7f5954072df0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f594800b580 tx=0x7f594800bfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f5951ffb700 1 -- 172.21.15.93:0/2087592927 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594801d070 con 0x7f59541004c0 2023-12-14T18:29:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f595abe8700 1 -- 172.21.15.93:0/2087592927 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59541102c0 con 0x7f59541004c0 2023-12-14T18:29:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.900+0000 7f595abe8700 1 -- 172.21.15.93:0/2087592927 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5954110130 con 0x7f59541004c0 2023-12-14T18:29:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.901+0000 7f5951ffb700 1 -- 172.21.15.93:0/2087592927 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f59480037d0 con 0x7f59541004c0 2023-12-14T18:29:10.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.901+0000 7f5951ffb700 1 -- 172.21.15.93:0/2087592927 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948017650 con 0x7f59541004c0 2023-12-14T18:29:10.912 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.903+0000 7f5951ffb700 1 -- 172.21.15.93:0/2087592927 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f594800fa90 con 0x7f59541004c0 2023-12-14T18:29:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.903+0000 7f5951ffb700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59440a1340 0x7f59440a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.904+0000 7f5951ffb700 1 -- 172.21.15.93:0/2087592927 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f59480cde70 con 0x7f59541004c0 2023-12-14T18:29:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.904+0000 7f5953fff700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59440a1340 0x7f59440a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.904+0000 7f5966d76b80 1 Processor -- start 2023-12-14T18:29:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.904+0000 7f5966d76b80 1 -- start start 2023-12-14T18:29:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5953fff700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59440a1340 0x7f59440a37f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59540745e0 tx=0x7f5940009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad784e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 0x55e7ad7857e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.913 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad7900f0 0x55e7ad7924d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5966d76b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad66f580 con 0x55e7ad7900f0 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5966d76b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad6de810 con 0x55e7ad5e2430 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5966d76b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad6d7ca0 con 0x55e7ad785370 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5958984700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad784e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5959185700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad7900f0 0x55e7ad7924d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5953fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 0x55e7ad7857e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5958984700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad784e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:51972/0 (socket says 172.21.15.93:51972) 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5958984700 1 -- 172.21.15.93:0/4099414216 learned_addr learned my addr 172.21.15.93:0/4099414216 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5958984700 1 -- 172.21.15.93:0/4099414216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 msgr2=0x55e7ad7857e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:10.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5958984700 1 --2- 172.21.15.93:0/4099414216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 0x55e7ad7857e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5958984700 1 -- 172.21.15.93:0/4099414216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad7900f0 msgr2=0x55e7ad7924d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5958984700 1 --2- 172.21.15.93:0/4099414216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad7900f0 0x55e7ad7924d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5958984700 1 -- 172.21.15.93:0/4099414216 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7ad559350 con 0x55e7ad5e2430 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.905+0000 7f5958984700 1 --2- 172.21.15.93:0/4099414216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad784e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f594800f690 tx=0x7f594809ad60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1b3276172f0486d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f593effd700 1 -- 172.21.15.93:0/4099414216 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948099080 con 0x55e7ad5e2430 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f593effd700 1 -- 172.21.15.93:0/4099414216 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f594809d020 con 0x55e7ad5e2430 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f593effd700 1 -- 172.21.15.93:0/4099414216 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59480a2c10 con 0x55e7ad5e2430 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f5966d76b80 1 -- 172.21.15.93:0/4099414216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 msgr2=0x55e7ad784e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f5966d76b80 1 --2- 172.21.15.93:0/4099414216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad784e30 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f594800f690 tx=0x7f594809ad60 comp rx=0 tx=0).stop 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f5966d76b80 1 -- 172.21.15.93:0/4099414216 shutdown_connections 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f5966d76b80 1 --2- 172.21.15.93:0/4099414216 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad7900f0 0x55e7ad7924d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f5966d76b80 1 --2- 172.21.15.93:0/4099414216 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad784e30 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f5966d76b80 1 --2- 172.21.15.93:0/4099414216 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 0x55e7ad7857e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f5966d76b80 1 -- 172.21.15.93:0/4099414216 >> 172.21.15.93:0/4099414216 conn(0x55e7ad77b890 msgr2=0x55e7ad781200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.906+0000 7f5966d76b80 1 -- 172.21.15.93:0/4099414216 shutdown_connections 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.907+0000 7f5966d76b80 1 -- 172.21.15.93:0/4099414216 wait complete. 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.907+0000 7f5966d76b80 1 -- start start 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.907+0000 7f5966d76b80 1 Processor -- start 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.907+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad5e2430 0x55e7ad78fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.907+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 0x55e7ad78cd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.907+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad78d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5966d76b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad673f30 con 0x55e7ad785370 2023-12-14T18:29:10.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5966d76b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad673bd0 con 0x55e7ad7900f0 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5966d76b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad55b640 con 0x55e7ad5e2430 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5966d76b80 10 client.0 _ll_register_callbacks cb 0x55e7ad6978b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5959185700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad78d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5958984700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad5e2430 0x55e7ad78fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5959185700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad78d270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:51974/0 (socket says 172.21.15.93:51974) 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5959185700 1 -- 172.21.15.93:0/434526740 learned_addr learned my addr 172.21.15.93:0/434526740 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5959185700 1 -- 172.21.15.93:0/434526740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad5e2430 msgr2=0x55e7ad78fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5959185700 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad5e2430 0x55e7ad78fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5959185700 1 -- 172.21.15.93:0/434526740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 msgr2=0x55e7ad78cd30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:29:10.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5959185700 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 0x55e7ad78cd30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5959185700 1 -- 172.21.15.93:0/434526740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5954110130 con 0x55e7ad7900f0 2023-12-14T18:29:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5958984700 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad5e2430 0x55e7ad78fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5959185700 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad78d270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f594c011ef0 tx=0x7f594c014600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2b66cc7426572559 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:10.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.908+0000 7f5933fff700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-12-14T18:29:10.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.913+0000 7f5933fff700 1 -- 172.21.15.93:0/434526740 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594c00e040 con 0x55e7ad7900f0 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.913+0000 7f5933fff700 1 -- 172.21.15.93:0/434526740 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f594c01a030 con 0x55e7ad7900f0 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.913+0000 7f5966d76b80 1 -- 172.21.15.93:0/434526740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55e7ad778ea0 con 0x55e7ad7900f0 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.913+0000 7f5933fff700 1 -- 172.21.15.93:0/434526740 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f594c0093e0 con 0x55e7ad7900f0 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.913+0000 7f5966d76b80 10 client.25373 fetch_fsmap learned FSMap version 27 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.913+0000 7f5966d76b80 1 -- 172.21.15.93:0/434526740 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e7ad69e110 con 0x55e7ad7900f0 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.913+0000 7f5933fff700 1 -- 172.21.15.93:0/434526740 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55e7ad778ea0 con 0x55e7ad7900f0 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.915+0000 7f5933fff700 1 -- 172.21.15.93:0/434526740 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f594c004540 con 0x55e7ad7900f0 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.915+0000 7f5966d76b80 10 client.25373 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.915+0000 7f5966d76b80 10 client.25373 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.915+0000 7f5966d76b80 20 client.25373 populate_metadata read hostname 'smithi093' 2023-12-14T18:29:10.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.915+0000 7f5966d76b80 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55e7ad79f9b0 0x55e7ad7a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.915+0000 7f5966d76b80 4 client.25373 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.915+0000 7f5966d76b80 1 -- 172.21.15.93:0/434526740 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55e7ad6d0c80 con 0x55e7ad79f9b0 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.915+0000 7f5958984700 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55e7ad79f9b0 0x55e7ad7a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.916+0000 7f5958984700 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55e7ad79f9b0 0x55e7ad7a1dc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e281316815a43a89 server_cookie=373a3131bdff8a67 in_seq=0 out_seq=0 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.916+0000 7f5933fff700 10 client.25373 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5933fff700 1 -- 172.21.15.93:0/434526740 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55e7ad6d0c80 con 0x55e7ad79f9b0 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5933fff700 10 client.25373 handle_command_reply: tid=0 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 client.25373 shutdown 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 -- 172.21.15.93:0/434526740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 msgr2=0x55e7ad78d270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad78d270 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f594c011ef0 tx=0x7f594c014600 comp rx=0 tx=0).stop 2023-12-14T18:29:10.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 -- 172.21.15.93:0/434526740 shutdown_connections 2023-12-14T18:29:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55e7ad79f9b0 0x55e7ad7a1dc0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad78d270 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad5e2430 0x55e7ad78fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 --2- 172.21.15.93:0/434526740 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 0x55e7ad78cd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5933fff700 0 client.25373 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 -- 172.21.15.93:0/434526740 >> 172.21.15.93:0/434526740 conn(0x55e7ad77b890 msgr2=0x55e7ad780d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 -- 172.21.15.93:0/434526740 shutdown_connections 2023-12-14T18:29:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 1 -- 172.21.15.93:0/434526740 wait complete. 2023-12-14T18:29:10.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.918+0000 7f5966d76b80 20 client.25373 trim_cache size 0 max 16384 2023-12-14T18:29:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5966d76b80 1 Processor -- start 2023-12-14T18:29:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5966d76b80 1 -- start start 2023-12-14T18:29:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad78c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 0x55e7ad78f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad7900f0 0x55e7ad78fa10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5966d76b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad7851f0 con 0x55e7ad785370 2023-12-14T18:29:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5966d76b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad7924d0 con 0x55e7ad5e2430 2023-12-14T18:29:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5966d76b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad78b290 con 0x55e7ad7900f0 2023-12-14T18:29:10.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5953fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 0x55e7ad78f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5958984700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad78c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5958984700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad78c4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:51984/0 (socket says 172.21.15.93:51984) 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5953fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 0x55e7ad78f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55470/0 (socket says 172.21.15.93:55470) 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.935+0000 7f5958984700 1 -- 172.21.15.93:0/1341000705 learned_addr learned my addr 172.21.15.93:0/1341000705 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f5959185700 1 --2- 172.21.15.93:0/1341000705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad7900f0 0x55e7ad78fa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f5958984700 1 -- 172.21.15.93:0/1341000705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad7900f0 msgr2=0x55e7ad78fa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f5958984700 1 --2- 172.21.15.93:0/1341000705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad7900f0 0x55e7ad78fa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f5958984700 1 -- 172.21.15.93:0/1341000705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 msgr2=0x55e7ad78f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f5958984700 1 --2- 172.21.15.93:0/1341000705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 0x55e7ad78f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f5958984700 1 -- 172.21.15.93:0/1341000705 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55e7ad785c60 con 0x55e7ad5e2430 2023-12-14T18:29:10.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f5958984700 1 --2- 172.21.15.93:0/1341000705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad78c4f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f59480cb8d0 tx=0x7f594801a040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d2f408bfdb2a8d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f593f7fe700 1 -- 172.21.15.93:0/1341000705 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594809ace0 con 0x55e7ad5e2430 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f593f7fe700 1 -- 172.21.15.93:0/1341000705 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f594809af40 con 0x55e7ad5e2430 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.936+0000 7f593f7fe700 1 -- 172.21.15.93:0/1341000705 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948096ad0 con 0x55e7ad5e2430 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 -- 172.21.15.93:0/1341000705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 msgr2=0x55e7ad78c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 --2- 172.21.15.93:0/1341000705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad78c4f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f59480cb8d0 tx=0x7f594801a040 comp rx=0 tx=0).stop 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 -- 172.21.15.93:0/1341000705 shutdown_connections 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 --2- 172.21.15.93:0/1341000705 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad7900f0 0x55e7ad78fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 --2- 172.21.15.93:0/1341000705 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad5e2430 0x55e7ad78c4f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 --2- 172.21.15.93:0/1341000705 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad785370 0x55e7ad78f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 -- 172.21.15.93:0/1341000705 >> 172.21.15.93:0/1341000705 conn(0x55e7ad77b890 msgr2=0x55e7ad780d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:10.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 -- 172.21.15.93:0/1341000705 shutdown_connections 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 -- 172.21.15.93:0/1341000705 wait complete. 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.937+0000 7f5966d76b80 1 -- start start 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5966d76b80 1 Processor -- start 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad5e2430 0x55e7ad794a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 0x55e7ad794fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5966d76b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad7f2870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5966d76b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad7958e0 con 0x55e7ad5e2430 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5966d76b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad7f2db0 con 0x55e7ad7900f0 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5966d76b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55e7ad7f2ef0 con 0x55e7ad785370 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5966d76b80 10 client.0 _ll_register_callbacks cb 0x55e7ad6978b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:10.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5958984700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad5e2430 0x55e7ad794a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5953fff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 0x55e7ad794fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5958984700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad5e2430 0x55e7ad794a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55482/0 (socket says 172.21.15.93:55482) 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5958984700 1 -- 172.21.15.93:0/449336230 learned_addr learned my addr 172.21.15.93:0/449336230 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5959185700 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad7f2870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5953fff700 1 -- 172.21.15.93:0/449336230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 msgr2=0x55e7ad7f2870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5953fff700 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad7f2870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5953fff700 1 -- 172.21.15.93:0/449336230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad5e2430 msgr2=0x55e7ad794a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5953fff700 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad5e2430 0x55e7ad794a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.938+0000 7f5953fff700 1 -- 172.21.15.93:0/449336230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55e7ad559350 con 0x55e7ad785370 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5958984700 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad5e2430 0x55e7ad794a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5953fff700 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 0x55e7ad794fc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55e7ad78e270 tx=0x7f59400047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8ed0eba0328ec9 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:10.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5933fff700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5933fff700 1 -- 172.21.15.93:0/449336230 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5940003ca0 con 0x55e7ad785370 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5933fff700 1 -- 172.21.15.93:0/449336230 <== mon.1 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f5940003e00 con 0x55e7ad785370 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5966d76b80 1 -- 172.21.15.93:0/449336230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f594809d020 con 0x55e7ad785370 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5933fff700 1 -- 172.21.15.93:0/449336230 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=27) v2 ==== 24+0+0 (secure 0 0 0) 0x7f594001d410 con 0x55e7ad785370 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5966d76b80 10 client.25372 fetch_fsmap learned FSMap version 27 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5966d76b80 1 -- 172.21.15.93:0/449336230 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=27,monmap=4+}) v3 -- 0x55e7ad505b00 con 0x55e7ad785370 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.939+0000 7f5933fff700 1 -- 172.21.15.93:0/449336230 <== mon.1 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f594809d020 con 0x55e7ad785370 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5933fff700 1 -- 172.21.15.93:0/449336230 <== mon.1 v2:172.21.15.160:3300/0 5 ==== fsmap(e 27) v1 ==== 7482+0+0 (secure 0 0 0) 0x7f5940027660 con 0x55e7ad785370 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5966d76b80 10 client.25372 fetch_fsmap finished waiting for FSMap version 27 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5966d76b80 10 client.25372 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:29:10.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5966d76b80 20 client.25372 populate_metadata read hostname 'smithi093' 2023-12-14T18:29:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5966d76b80 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55e7ad7f4710 0x55e7ad7f6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5966d76b80 4 client.25372 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:29:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5966d76b80 1 -- 172.21.15.93:0/449336230 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55e7ad6d09a0 con 0x55e7ad7f4710 2023-12-14T18:29:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5958984700 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55e7ad7f4710 0x55e7ad7f6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5933fff700 10 client.25372 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:10.949 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:10.940+0000 7f5958984700 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55e7ad7f4710 0x55e7ad7f6b20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81a9773e33038277 server_cookie=1e91dce6fa3470c8 in_seq=0 out_seq=0 2023-12-14T18:29:11.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:10 smithi160 ceph-mon[43840]: pgmap v84: 129 pgs: 129 active+clean; 57 MiB data, 216 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 470 op/s 2023-12-14T18:29:11.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:10 smithi093 ceph-mon[42070]: pgmap v84: 129 pgs: 129 active+clean; 57 MiB data, 216 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 470 op/s 2023-12-14T18:29:11.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:10 smithi097 ceph-mon[43486]: pgmap v84: 129 pgs: 129 active+clean; 57 MiB data, 216 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.8 MiB/s wr, 470 op/s 2023-12-14T18:29:13.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:12 smithi160 ceph-mon[43840]: pgmap v85: 129 pgs: 129 active+clean; 57 MiB data, 216 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 30 op/s 2023-12-14T18:29:13.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:12 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:29:13.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:12 smithi093 ceph-mon[42070]: pgmap v85: 129 pgs: 129 active+clean; 57 MiB data, 216 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 30 op/s 2023-12-14T18:29:13.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:12 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:29:13.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:12 smithi097 ceph-mon[43486]: pgmap v85: 129 pgs: 129 active+clean; 57 MiB data, 216 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.0 MiB/s wr, 30 op/s 2023-12-14T18:29:13.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:12 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:29:13.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.807+0000 7f5933fff700 1 -- 172.21.15.93:0/449336230 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55e7ad6d09a0 con 0x55e7ad7f4710 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.807+0000 7f5933fff700 10 client.25372 handle_command_reply: tid=0 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 client.25372 shutdown 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 -- 172.21.15.93:0/449336230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 msgr2=0x55e7ad794fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 0x55e7ad794fc0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x55e7ad78e270 tx=0x7f59400047c0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 -- 172.21.15.93:0/449336230 shutdown_connections 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55e7ad7f4710 0x55e7ad7f6b20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55e7ad5e2430 0x55e7ad794a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55e7ad7900f0 0x55e7ad7f2870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 --2- 172.21.15.93:0/449336230 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55e7ad785370 0x55e7ad794fc0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5933fff700 0 client.25372 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:13.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 -- 172.21.15.93:0/449336230 >> 172.21.15.93:0/449336230 conn(0x55e7ad77b890 msgr2=0x55e7ad79b240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 -- 172.21.15.93:0/449336230 shutdown_connections 2023-12-14T18:29:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 1 -- 172.21.15.93:0/449336230 wait complete. 2023-12-14T18:29:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.808+0000 7f5966d76b80 20 client.25372 trim_cache size 0 max 16384 2023-12-14T18:29:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:29:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:29:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:29:13.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:29:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 -- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59440a1340 msgr2=0x7f59440a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59440a1340 0x7f59440a37f0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f59540745e0 tx=0x7f5940009260 comp rx=0 tx=0).stop 2023-12-14T18:29:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 -- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 msgr2=0x7f5954072df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:13.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 0x7f5954072df0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f594800b580 tx=0x7f594800bfd0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 -- 172.21.15.93:0/2087592927 shutdown_connections 2023-12-14T18:29:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f59440a1340 0x7f59440a37f0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f59541004c0 0x7f5954072df0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5954105500 0x7f59541a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 --2- 172.21.15.93:0/2087592927 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f5954100e60 0x7f5954073330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 -- 172.21.15.93:0/2087592927 >> 172.21.15.93:0/2087592927 conn(0x7f59540fbd50 msgr2=0x7f59540fc850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 -- 172.21.15.93:0/2087592927 shutdown_connections 2023-12-14T18:29:13.814 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:13.809+0000 7f595abe8700 1 -- 172.21.15.93:0/2087592927 wait complete. 2023-12-14T18:29:15.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:14 smithi160 ceph-mon[43840]: pgmap v86: 129 pgs: 129 active+clean; 75 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.9 MiB/s wr, 1.24k op/s 2023-12-14T18:29:15.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:14 smithi093 ceph-mon[42070]: pgmap v86: 129 pgs: 129 active+clean; 75 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.9 MiB/s wr, 1.24k op/s 2023-12-14T18:29:15.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:14 smithi097 ceph-mon[43486]: pgmap v86: 129 pgs: 129 active+clean; 75 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 5.9 MiB/s wr, 1.24k op/s 2023-12-14T18:29:17.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:16 smithi160 ceph-mon[43840]: pgmap v87: 129 pgs: 129 active+clean; 75 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 1.22k op/s 2023-12-14T18:29:17.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:16 smithi093 ceph-mon[42070]: pgmap v87: 129 pgs: 129 active+clean; 75 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 1.22k op/s 2023-12-14T18:29:17.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:16 smithi097 ceph-mon[43486]: pgmap v87: 129 pgs: 129 active+clean; 75 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 1.22k op/s 2023-12-14T18:29:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:18 smithi160 ceph-mon[43840]: pgmap v88: 129 pgs: 129 active+clean; 75 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 1.22k op/s 2023-12-14T18:29:19.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:18 smithi093 ceph-mon[42070]: pgmap v88: 129 pgs: 129 active+clean; 75 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 1.22k op/s 2023-12-14T18:29:19.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:18 smithi097 ceph-mon[43486]: pgmap v88: 129 pgs: 129 active+clean; 75 MiB data, 289 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.9 MiB/s wr, 1.22k op/s 2023-12-14T18:29:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:20 smithi160 ceph-mon[43840]: pgmap v89: 129 pgs: 129 active+clean; 82 MiB data, 311 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 1.24k op/s 2023-12-14T18:29:21.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:20 smithi093 ceph-mon[42070]: pgmap v89: 129 pgs: 129 active+clean; 82 MiB data, 311 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 1.24k op/s 2023-12-14T18:29:21.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:20 smithi097 ceph-mon[43486]: pgmap v89: 129 pgs: 129 active+clean; 82 MiB data, 311 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 5.9 MiB/s wr, 1.24k op/s 2023-12-14T18:29:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:22 smithi160 ceph-mon[43840]: pgmap v90: 129 pgs: 129 active+clean; 82 MiB data, 311 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 1.23k op/s 2023-12-14T18:29:23.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:22 smithi093 ceph-mon[42070]: pgmap v90: 129 pgs: 129 active+clean; 82 MiB data, 311 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 1.23k op/s 2023-12-14T18:29:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:22 smithi097 ceph-mon[43486]: pgmap v90: 129 pgs: 129 active+clean; 82 MiB data, 311 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 MiB/s rd, 4.0 MiB/s wr, 1.23k op/s 2023-12-14T18:29:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:24 smithi160 ceph-mon[43840]: pgmap v91: 129 pgs: 129 active+clean; 113 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.6 MiB/s wr, 1.48k op/s 2023-12-14T18:29:25.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:24 smithi093 ceph-mon[42070]: pgmap v91: 129 pgs: 129 active+clean; 113 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.6 MiB/s wr, 1.48k op/s 2023-12-14T18:29:25.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:24 smithi097 ceph-mon[43486]: pgmap v91: 129 pgs: 129 active+clean; 113 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 7.6 MiB/s wr, 1.48k op/s 2023-12-14T18:29:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.923+0000 7fb2a289e700 1 -- 172.21.15.93:0/3847471324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101440 msgr2=0x7fb29c101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:26.928 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.923+0000 7fb2a289e700 1 --2- 172.21.15.93:0/3847471324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101440 0x7fb29c101810 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7fb284009990 tx=0x7fb284009ca0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.923+0000 7fb2a289e700 1 -- 172.21.15.93:0/3847471324 shutdown_connections 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.923+0000 7fb2a289e700 1 --2- 172.21.15.93:0/3847471324 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101440 0x7fb29c101810 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.923+0000 7fb2a289e700 1 --2- 172.21.15.93:0/3847471324 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c10a810 0x7fb29c10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.923+0000 7fb2a289e700 1 --2- 172.21.15.93:0/3847471324 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29c101d50 0x7fb29c10a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.923+0000 7fb2a289e700 1 -- 172.21.15.93:0/3847471324 >> 172.21.15.93:0/3847471324 conn(0x7fb29c0763a0 msgr2=0x7fb29c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.924+0000 7fb2a289e700 1 -- 172.21.15.93:0/3847471324 shutdown_connections 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.924+0000 7fb2a289e700 1 -- 172.21.15.93:0/3847471324 wait complete. 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.924+0000 7fb2a289e700 1 Processor -- start 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.925+0000 7fb2a289e700 1 -- start start 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.925+0000 7fb2a289e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c101440 0x7fb29c108ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.929 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.925+0000 7fb2a289e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101d50 0x7fb29c109400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.925+0000 7fb2a289e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29c10a810 0x7fb29c198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.925+0000 7fb2a289e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c1991a0 con 0x7fb29c101d50 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.925+0000 7fb2a289e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c199310 con 0x7fb29c101440 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.925+0000 7fb2a289e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb29c1994d0 con 0x7fb29c10a810 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.926+0000 7fb29b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101d50 0x7fb29c109400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.926+0000 7fb29b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101d50 0x7fb29c109400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37962/0 (socket says 172.21.15.93:37962) 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.926+0000 7fb29b7fe700 1 -- 172.21.15.93:0/2973727532 learned_addr learned my addr 172.21.15.93:0/2973727532 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.926+0000 7fb2a0e3b700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29c10a810 0x7fb29c198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.926+0000 7fb29b7fe700 1 -- 172.21.15.93:0/2973727532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29c10a810 msgr2=0x7fb29c198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.926+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29c10a810 0x7fb29c198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.926+0000 7fb29b7fe700 1 -- 172.21.15.93:0/2973727532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c101440 msgr2=0x7fb29c108ec0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.926+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c101440 0x7fb29c108ec0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.930 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.926+0000 7fb29b7fe700 1 -- 172.21.15.93:0/2973727532 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2840095f0 con 0x7fb29c101d50 2023-12-14T18:29:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.927+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101d50 0x7fb29c109400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00e9a0 tx=0x7fb28c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.927+0000 7fb2997fa700 1 -- 172.21.15.93:0/2973727532 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c00cbd0 con 0x7fb29c101d50 2023-12-14T18:29:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.927+0000 7fb2a289e700 1 -- 172.21.15.93:0/2973727532 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb29c110310 con 0x7fb29c101d50 2023-12-14T18:29:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.927+0000 7fb2997fa700 1 -- 172.21.15.93:0/2973727532 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c00cd30 con 0x7fb29c101d50 2023-12-14T18:29:26.940 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.927+0000 7fb2997fa700 1 -- 172.21.15.93:0/2973727532 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c018680 con 0x7fb29c101d50 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.927+0000 7fb2a289e700 1 -- 172.21.15.93:0/2973727532 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb29c110180 con 0x7fb29c101d50 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.929+0000 7fb2997fa700 1 -- 172.21.15.93:0/2973727532 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb28c0187e0 con 0x7fb29c101d50 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.930+0000 7fb2997fa700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb2880a1300 0x7fb2880a37b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.930+0000 7fb2997fa700 1 -- 172.21.15.93:0/2973727532 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb28c014070 con 0x7fb29c101d50 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.930+0000 7fb29bfff700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb2880a1300 0x7fb2880a37b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.930+0000 7fb29bfff700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb2880a1300 0x7fb2880a37b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb284006010 tx=0x7fb284000bc0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2aea2cb80 1 Processor -- start 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2aea2cb80 1 -- start start 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x5635848f5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 0x5635848f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635849034d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.941 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635847e0580 con 0x5635849010f0 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56358484f810 con 0x5635848f6370 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563584848ca0 con 0x563584753430 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2a0e3b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635849034d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2a0e3b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635849034d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37978/0 (socket says 172.21.15.93:37978) 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2a0e3b700 1 -- 172.21.15.93:0/265330371 learned_addr learned my addr 172.21.15.93:0/265330371 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb29bfff700 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x5635848f5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 0x5635848f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2a0e3b700 1 -- 172.21.15.93:0/265330371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 msgr2=0x5635848f5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2a0e3b700 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x5635848f5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2a0e3b700 1 -- 172.21.15.93:0/265330371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 msgr2=0x5635848f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2a0e3b700 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 0x5635848f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2a0e3b700 1 -- 172.21.15.93:0/265330371 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5635846ca350 con 0x5635849010f0 2023-12-14T18:29:26.942 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb29bfff700 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x5635848f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.931+0000 7fb2a0e3b700 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635849034d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d50 tx=0x7fb2940077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33010dbc4856259b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2927fc700 1 -- 172.21.15.93:0/265330371 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294018070 con 0x5635849010f0 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2927fc700 1 -- 172.21.15.93:0/265330371 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29400f050 con 0x5635849010f0 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2927fc700 1 -- 172.21.15.93:0/265330371 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294014410 con 0x5635849010f0 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/265330371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 msgr2=0x5635849034d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635849034d0 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d50 tx=0x7fb2940077e0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/265330371 shutdown_connections 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635849034d0 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x5635848f5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/265330371 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 0x5635848f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.932+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/265330371 >> 172.21.15.93:0/265330371 conn(0x5635848ec890 msgr2=0x5635848f2200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:26.943 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/265330371 shutdown_connections 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/265330371 wait complete. 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 -- start start 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 Processor -- start 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584900a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x5635848fdd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 0x5635848fe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635847e4f30 con 0x5635848f6370 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635847e4bd0 con 0x5635849010f0 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635846cc640 con 0x563584753430 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.933+0000 7fb2aea2cb80 10 client.0 _ll_register_callbacks cb 0x5635848088b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x5635848fdd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584900a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.944 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x5635848fdd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:37984/0 (socket says 172.21.15.93:37984) 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29bfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584900a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:58388/0 (socket says 172.21.15.93:58388) 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29b7fe700 1 -- 172.21.15.93:0/1528959783 learned_addr learned my addr 172.21.15.93:0/1528959783 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb2a0e3b700 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 0x5635848fe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29b7fe700 1 -- 172.21.15.93:0/1528959783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 msgr2=0x563584900a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584900a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29b7fe700 1 -- 172.21.15.93:0/1528959783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 msgr2=0x5635848fe300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 0x5635848fe300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29b7fe700 1 -- 172.21.15.93:0/1528959783 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb294009980 con 0x5635848f6370 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x5635848fdd40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb28c099ec0 tx=0x7fb28c09a240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3a56a5b7346f0e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb27b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1528959783 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0998b0 con 0x5635848f6370 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1528959783 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb28c0a2610 con 0x5635848f6370 2023-12-14T18:29:26.945 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1528959783 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5635848e9ea0 con 0x5635848f6370 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1528959783 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb28c0a2180 con 0x5635848f6370 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb2aea2cb80 10 client.15768 fetch_fsmap learned FSMap version 28 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1528959783 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x56358480f110 con 0x5635848f6370 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.934+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1528959783 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb28c0c8020 con 0x5635848f6370 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1528959783 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb28c099120 con 0x5635848f6370 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb2aea2cb80 10 client.15768 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb2aea2cb80 10 client.15768 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb2aea2cb80 20 client.15768 populate_metadata read hostname 'smithi093' 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563584910a90 0x563584912ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb2aea2cb80 4 client.15768 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1528959783 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563584841c80 con 0x563584910a90 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb29bfff700 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563584910a90 0x563584912ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb27b7fe700 10 client.15768 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:26.946 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.935+0000 7fb29bfff700 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563584910a90 0x563584912ea0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c584cc270cde535e server_cookie=21948713360e6a3c in_seq=0 out_seq=0 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.937+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1528959783 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563584841c80 con 0x563584910a90 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.937+0000 7fb27b7fe700 10 client.15768 handle_command_reply: tid=0 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.937+0000 7fb2aea2cb80 1 client.15768 shutdown 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1528959783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 msgr2=0x5635848fdd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x5635848fdd40 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fb28c099ec0 tx=0x7fb28c09a240 comp rx=0 tx=0).stop 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1528959783 shutdown_connections 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563584910a90 0x563584912ea0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 0x5635848fe300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb27b7fe700 0 client.15768 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584900a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1528959783 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x5635848fdd40 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1528959783 >> 172.21.15.93:0/1528959783 conn(0x5635848ec890 msgr2=0x5635848f1d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1528959783 shutdown_connections 2023-12-14T18:29:26.947 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1528959783 wait complete. 2023-12-14T18:29:26.948 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.938+0000 7fb2aea2cb80 20 client.15768 trim_cache size 0 max 16384 2023-12-14T18:29:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.952+0000 7fb2aea2cb80 1 Processor -- start 2023-12-14T18:29:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.952+0000 7fb2aea2cb80 1 -- start start 2023-12-14T18:29:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.952+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584900550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.952+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 0x563584900a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.952+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635848fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.952+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635848f7930 con 0x5635849010f0 2023-12-14T18:29:26.956 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.952+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635848fc2a0 con 0x5635848f6370 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.952+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635848f0fb0 con 0x563584753430 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb2a0e3b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635848fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb2a0e3b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635848fdd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38000/0 (socket says 172.21.15.93:38000) 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb2a0e3b700 1 -- 172.21.15.93:0/1982838568 learned_addr learned my addr 172.21.15.93:0/1982838568 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb29bfff700 1 --2- 172.21.15.93:0/1982838568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584900550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/1982838568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 0x563584900a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb2a0e3b700 1 -- 172.21.15.93:0/1982838568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 msgr2=0x563584900550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb2a0e3b700 1 --2- 172.21.15.93:0/1982838568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584900550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb2a0e3b700 1 -- 172.21.15.93:0/1982838568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 msgr2=0x563584900a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb2a0e3b700 1 --2- 172.21.15.93:0/1982838568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 0x563584900a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb2a0e3b700 1 -- 172.21.15.93:0/1982838568 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56358480ee60 con 0x5635849010f0 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb2a0e3b700 1 --2- 172.21.15.93:0/1982838568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635848fdd00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d50 tx=0x7fb2940060f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02114bb572fb926 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:26.957 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb292ffd700 1 -- 172.21.15.93:0/1982838568 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294018030 con 0x5635849010f0 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb292ffd700 1 -- 172.21.15.93:0/1982838568 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb294019040 con 0x5635849010f0 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.953+0000 7fb292ffd700 1 -- 172.21.15.93:0/1982838568 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb294009ea0 con 0x5635849010f0 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1982838568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 msgr2=0x5635848fdd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1982838568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635848fdd00 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fb294009d50 tx=0x7fb2940060f0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1982838568 shutdown_connections 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1982838568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635849010f0 0x5635848fdd00 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1982838568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584900550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1982838568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635848f6370 0x563584900a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1982838568 >> 172.21.15.93:0/1982838568 conn(0x5635848ec890 msgr2=0x56358490c430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1982838568 shutdown_connections 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1982838568 wait complete. 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 -- start start 2023-12-14T18:29:26.958 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.954+0000 7fb2aea2cb80 1 Processor -- start 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584907b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x563584905e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 0x563584906360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5635849068a0 con 0x5635848f6370 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563584906a10 con 0x5635849010f0 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563584906bd0 con 0x563584753430 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 10 client.0 _ll_register_callbacks cb 0x7fb28c099b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x563584905e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29b7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x563584905e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:38016/0 (socket says 172.21.15.93:38016) 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29b7fe700 1 -- 172.21.15.93:0/1503879510 learned_addr learned my addr 172.21.15.93:0/1503879510 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:26.961 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29bfff700 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584907b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2a0e3b700 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 0x563584906360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29b7fe700 1 -- 172.21.15.93:0/1503879510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 msgr2=0x563584907b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584907b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29b7fe700 1 -- 172.21.15.93:0/1503879510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 msgr2=0x563584906360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 0x563584906360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29b7fe700 1 -- 172.21.15.93:0/1503879510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5635846ca350 con 0x5635848f6370 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb29b7fe700 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x563584905e20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb28c09a150 tx=0x7fb28c0a23b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b89fcd25242e47e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb27b7fe700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1503879510 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb28c0a2610 con 0x5635848f6370 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1503879510 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb28c0c9080 con 0x5635848f6370 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1503879510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb29400f050 con 0x5635848f6370 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1503879510 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb28c0b2ba0 con 0x5635848f6370 2023-12-14T18:29:26.962 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 10 client.15774 fetch_fsmap learned FSMap version 28 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.955+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1503879510 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5635847f5940 con 0x5635848f6370 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1503879510 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb29400f050 con 0x5635848f6370 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1503879510 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fb28c095fb0 con 0x5635848f6370 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb2aea2cb80 10 client.15774 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb2aea2cb80 10 client.15774 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb2aea2cb80 20 client.15774 populate_metadata read hostname 'smithi093' 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563584965980 0x563584967d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb2aea2cb80 4 client.15774 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1503879510 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x5635848419a0 con 0x563584965980 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb29bfff700 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563584965980 0x563584967d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb27b7fe700 10 client.15774 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:26.963 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:26.956+0000 7fb29bfff700 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563584965980 0x563584967d60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f2dffb92d2950c3 server_cookie=ea5384ed792db7a6 in_seq=0 out_seq=0 2023-12-14T18:29:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:26 smithi160 ceph-mon[43840]: pgmap v92: 129 pgs: 129 active+clean; 113 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2023-12-14T18:29:27.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:26 smithi093 ceph-mon[42070]: pgmap v92: 129 pgs: 129 active+clean; 113 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2023-12-14T18:29:27.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:26 smithi097 ceph-mon[43486]: pgmap v92: 129 pgs: 129 active+clean; 113 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2023-12-14T18:29:29.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:28 smithi160 ceph-mon[43840]: pgmap v93: 129 pgs: 129 active+clean; 113 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2023-12-14T18:29:29.344 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:28 smithi093 ceph-mon[42070]: pgmap v93: 129 pgs: 129 active+clean; 113 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2023-12-14T18:29:29.348 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:28 smithi097 ceph-mon[43486]: pgmap v93: 129 pgs: 129 active+clean; 113 MiB data, 394 MiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 5.7 MiB/s wr, 273 op/s 2023-12-14T18:29:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb27b7fe700 1 -- 172.21.15.93:0/1503879510 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x5635848419a0 con 0x563584965980 2023-12-14T18:29:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb27b7fe700 10 client.15774 handle_command_reply: tid=0 2023-12-14T18:29:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 client.15774 shutdown 2023-12-14T18:29:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1503879510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 msgr2=0x563584905e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x563584905e20 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fb28c09a150 tx=0x7fb28c0a23b0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1503879510 shutdown_connections 2023-12-14T18:29:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563584965980 0x563584967d60 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x5635849010f0 0x563584906360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.285 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563584753430 0x563584907b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb27b7fe700 0 client.15774 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 --2- 172.21.15.93:0/1503879510 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x5635848f6370 0x563584905e20 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1503879510 >> 172.21.15.93:0/1503879510 conn(0x5635848ec890 msgr2=0x5635848fdce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1503879510 shutdown_connections 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 1 -- 172.21.15.93:0/1503879510 wait complete. 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.282+0000 7fb2aea2cb80 20 client.15774 trim_cache size 0 max 16384 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:29:30.286 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:29:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 -- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb2880a1300 msgr2=0x7fb2880a37b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb2880a1300 0x7fb2880a37b0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb284006010 tx=0x7fb284000bc0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 -- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101d50 msgr2=0x7fb29c109400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:30.287 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101d50 0x7fb29c109400 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7fb28c00e9a0 tx=0x7fb28c00ed60 comp rx=0 tx=0).stop 2023-12-14T18:29:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 -- 172.21.15.93:0/2973727532 shutdown_connections 2023-12-14T18:29:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb29c10a810 0x7fb29c198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb29c101440 0x7fb29c108ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb2880a1300 0x7fb2880a37b0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 --2- 172.21.15.93:0/2973727532 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb29c101d50 0x7fb29c109400 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 -- 172.21.15.93:0/2973727532 >> 172.21.15.93:0/2973727532 conn(0x7fb29c0763a0 msgr2=0x7fb29c0ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 -- 172.21.15.93:0/2973727532 shutdown_connections 2023-12-14T18:29:30.288 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:30.283+0000 7fb2a289e700 1 -- 172.21.15.93:0/2973727532 wait complete. 2023-12-14T18:29:31.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:30 smithi160 ceph-mon[43840]: pgmap v94: 129 pgs: 129 active+clean; 137 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.9 MiB/s wr, 1.08k op/s 2023-12-14T18:29:31.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:30 smithi093 ceph-mon[42070]: pgmap v94: 129 pgs: 129 active+clean; 137 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.9 MiB/s wr, 1.08k op/s 2023-12-14T18:29:31.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:30 smithi097 ceph-mon[43486]: pgmap v94: 129 pgs: 129 active+clean; 137 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 8.9 MiB/s wr, 1.08k op/s 2023-12-14T18:29:33.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:32 smithi160 ceph-mon[43840]: pgmap v95: 129 pgs: 129 active+clean; 137 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 1.06k op/s 2023-12-14T18:29:33.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:32 smithi093 ceph-mon[42070]: pgmap v95: 129 pgs: 129 active+clean; 137 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 1.06k op/s 2023-12-14T18:29:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:32 smithi097 ceph-mon[43486]: pgmap v95: 129 pgs: 129 active+clean; 137 MiB data, 492 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 1.06k op/s 2023-12-14T18:29:35.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:34 smithi160 ceph-mon[43840]: pgmap v96: 129 pgs: 129 active+clean; 137 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 1.09k op/s 2023-12-14T18:29:35.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:34 smithi093 ceph-mon[42070]: pgmap v96: 129 pgs: 129 active+clean; 137 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 1.09k op/s 2023-12-14T18:29:35.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:34 smithi097 ceph-mon[43486]: pgmap v96: 129 pgs: 129 active+clean; 137 MiB data, 495 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 6.8 MiB/s wr, 1.09k op/s 2023-12-14T18:29:35.984 INFO:tasks.workunit.client.0.smithi093.stdout:created hierarchy, now cleaning up 2023-12-14T18:29:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:36 smithi160 ceph-mon[43840]: pgmap v97: 129 pgs: 129 active+clean; 141 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 2.03k op/s 2023-12-14T18:29:37.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:36 smithi093 ceph-mon[42070]: pgmap v97: 129 pgs: 129 active+clean; 141 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 2.03k op/s 2023-12-14T18:29:37.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:36 smithi097 ceph-mon[43486]: pgmap v97: 129 pgs: 129 active+clean; 141 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 2.03k op/s 2023-12-14T18:29:39.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:38 smithi160 ceph-mon[43840]: pgmap v98: 129 pgs: 129 active+clean; 141 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 2.03k op/s 2023-12-14T18:29:39.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:38 smithi093 ceph-mon[42070]: pgmap v98: 129 pgs: 129 active+clean; 141 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 2.03k op/s 2023-12-14T18:29:39.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:38 smithi097 ceph-mon[43486]: pgmap v98: 129 pgs: 129 active+clean; 141 MiB data, 590 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 5.4 MiB/s wr, 2.03k op/s 2023-12-14T18:29:41.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:40 smithi160 ceph-mon[43840]: pgmap v99: 129 pgs: 129 active+clean; 154 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.5 MiB/s wr, 2.04k op/s 2023-12-14T18:29:41.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:40 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:29:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:40 smithi093 ceph-mon[42070]: pgmap v99: 129 pgs: 129 active+clean; 154 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.5 MiB/s wr, 2.04k op/s 2023-12-14T18:29:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:40 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:29:41.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:40 smithi097 ceph-mon[43486]: pgmap v99: 129 pgs: 129 active+clean; 154 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 6.5 MiB/s wr, 2.04k op/s 2023-12-14T18:29:41.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:40 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:29:42.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:42 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:29:42.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:42 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:29:42.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:42 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:29:42.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:42 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:29:42.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:42 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:29:42.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:42 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:29:42.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:42 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:29:42.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:42 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:29:42.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:42 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:29:42.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:42 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:29:42.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:42 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:29:42.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:42 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:29:42.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:42 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:29:42.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:42 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:29:42.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:42 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:29:42.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:42 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:29:42.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:42 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:29:42.345 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:42 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:29:42.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:42 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:29:42.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:42 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:29:42.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:42 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:29:42.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:42 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:29:42.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:42 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:29:42.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:42 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:29:42.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:42 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:29:42.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:42 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:29:42.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:42 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:29:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.419+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/2956191077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 msgr2=0x7fcdb810be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.419+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/2956191077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 0x7fcdb810be50 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fcdb400b0d0 tx=0x7fcdb400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.419+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/2956191077 shutdown_connections 2023-12-14T18:29:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.419+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/2956191077 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 0x7fcdb810be50 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.419+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/2956191077 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdb81004c0 0x7fcdb8100890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.423 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.419+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/2956191077 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb8100dd0 0x7fcdb8109530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.419+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/2956191077 >> 172.21.15.93:0/2956191077 conn(0x7fcdb8074960 msgr2=0x7fcdb8074d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.419+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/2956191077 shutdown_connections 2023-12-14T18:29:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.419+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/2956191077 wait complete. 2023-12-14T18:29:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbfe2e700 1 Processor -- start 2023-12-14T18:29:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbfe2e700 1 -- start start 2023-12-14T18:29:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbfe2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdb81004c0 0x7fcdb8196370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbfe2e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb8100dd0 0x7fcdb81968b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.424 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbfe2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 0x7fcdb819ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbfe2e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb819b350 con 0x7fcdb8109a70 2023-12-14T18:29:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbfe2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb819b4c0 con 0x7fcdb81004c0 2023-12-14T18:29:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbfe2e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdb819b680 con 0x7fcdb8100dd0 2023-12-14T18:29:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbe3cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 0x7fcdb819ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbdbca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdb81004c0 0x7fcdb8196370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbe3cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 0x7fcdb819ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33478/0 (socket says 172.21.15.93:33478) 2023-12-14T18:29:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbdbca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdb81004c0 0x7fcdb8196370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:53536/0 (socket says 172.21.15.93:53536) 2023-12-14T18:29:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/251217205 learned_addr learned my addr 172.21.15.93:0/251217205 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:43.425 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbd3c9700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb8100dd0 0x7fcdb81968b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/251217205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb8100dd0 msgr2=0x7fcdb81968b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb8100dd0 0x7fcdb81968b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/251217205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdb81004c0 msgr2=0x7fcdb8196370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdb81004c0 0x7fcdb8196370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/251217205 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdb4009d00 con 0x7fcdb8109a70 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 0x7fcdb819ad10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4007870 tx=0x7fcdb40078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.420+0000 7fcdaeffd700 1 -- 172.21.15.93:0/251217205 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb400e040 con 0x7fcdb8109a70 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.421+0000 7fcdaeffd700 1 -- 172.21.15.93:0/251217205 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdb4005070 con 0x7fcdb8109a70 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.421+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/251217205 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdb810f150 con 0x7fcdb8109a70 2023-12-14T18:29:43.426 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.421+0000 7fcdaeffd700 1 -- 172.21.15.93:0/251217205 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb401b660 con 0x7fcdb8109a70 2023-12-14T18:29:43.427 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.421+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/251217205 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb810efc0 con 0x7fcdb8109a70 2023-12-14T18:29:43.434 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.422+0000 7fcdaeffd700 1 -- 172.21.15.93:0/251217205 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fcdb40051e0 con 0x7fcdb8109a70 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.423+0000 7fcdaeffd700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcda40a12c0 0x7fcda40a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.423+0000 7fcdbdbca700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcda40a12c0 0x7fcda40a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.423+0000 7fcdaeffd700 1 -- 172.21.15.93:0/251217205 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fcdb40ce080 con 0x7fcdb8109a70 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.424+0000 7fcdbdbca700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcda40a12c0 0x7fcda40a3770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8067590 tx=0x7fcdb0009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.424+0000 7fcdcbfbcb80 1 Processor -- start 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.424+0000 7fcdcbfbcb80 1 -- start start 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c336430 0x561c9c4d8e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 0x561c9c4d97e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c3c3580 con 0x561c9c4e40f0 2023-12-14T18:29:43.435 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c432810 con 0x561c9c336430 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c42bca0 con 0x561c9c4d9370 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbe3cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbdbca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c336430 0x561c9c4d8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbe3cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e64d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33484/0 (socket says 172.21.15.93:33484) 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbd3c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 0x561c9c4d97e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbdbca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c336430 0x561c9c4d8e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:53548/0 (socket says 172.21.15.93:53548) 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/2100689411 learned_addr learned my addr 172.21.15.93:0/2100689411 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/2100689411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 msgr2=0x561c9c4d97e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/2100689411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 0x561c9c4d97e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.436 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/2100689411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c336430 msgr2=0x561c9c4d8e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/2100689411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c336430 0x561c9c4d8e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/2100689411 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c9c2ad350 con 0x561c9c4e40f0 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/2100689411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e64d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcdb409ae80 tx=0x7fcdb400b3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4133508d9b0efae1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcda3fff700 1 -- 172.21.15.93:0/2100689411 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb409aef0 con 0x561c9c4e40f0 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcda3fff700 1 -- 172.21.15.93:0/2100689411 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdb409ef20 con 0x561c9c4e40f0 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.425+0000 7fcda3fff700 1 -- 172.21.15.93:0/2100689411 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb409b3b0 con 0x561c9c4e40f0 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2100689411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 msgr2=0x561c9c4e64d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2100689411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e64d0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fcdb409ae80 tx=0x7fcdb400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2100689411 shutdown_connections 2023-12-14T18:29:43.437 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2100689411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e64d0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2100689411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c336430 0x561c9c4d8e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2100689411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 0x561c9c4d97e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2100689411 >> 172.21.15.93:0/2100689411 conn(0x561c9c4cf890 msgr2=0x561c9c4d5200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2100689411 shutdown_connections 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2100689411 wait complete. 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 -- start start 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.426+0000 7fcdcbfbcb80 1 Processor -- start 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 0x561c9c4e3a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4d9370 0x561c9c4e0d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.438 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c3c7f30 con 0x561c9c4e40f0 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c3c7bd0 con 0x561c9c4d9370 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c2af640 con 0x561c9c336430 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdcbfbcb80 10 client.0 _ll_register_callbacks cb 0x561c9c3eb8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbe3cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbe3cb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33490/0 (socket says 172.21.15.93:33490) 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbd3c9700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4d9370 0x561c9c4e0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/1399891051 learned_addr learned my addr 172.21.15.93:0/1399891051 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbdbca700 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 0x561c9c4e3a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.439 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/1399891051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 msgr2=0x561c9c4e3a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 0x561c9c4e3a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/1399891051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4d9370 msgr2=0x561c9c4e0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4d9370 0x561c9c4e0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/1399891051 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdb810efc0 con 0x561c9c4e40f0 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e13b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40a2d80 tx=0x7fcdb40a3320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f05f9edf69f413ce server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.427+0000 7fcda17fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcda17fa700 1 -- 172.21.15.93:0/1399891051 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb409a5e0 con 0x561c9c4e40f0 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcda17fa700 1 -- 172.21.15.93:0/1399891051 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fcdb409b150 con 0x561c9c4e40f0 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/1399891051 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c9c4ccea0 con 0x561c9c4e40f0 2023-12-14T18:29:43.440 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcda17fa700 1 -- 172.21.15.93:0/1399891051 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb40b9250 con 0x561c9c4e40f0 2023-12-14T18:29:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdcbfbcb80 10 client.15786 fetch_fsmap learned FSMap version 28 2023-12-14T18:29:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/1399891051 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561c9c3f2110 con 0x561c9c4e40f0 2023-12-14T18:29:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcda17fa700 1 -- 172.21.15.93:0/1399891051 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c9c4ccea0 con 0x561c9c4e40f0 2023-12-14T18:29:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcda17fa700 1 -- 172.21.15.93:0/1399891051 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcdb40a2e20 con 0x561c9c4e40f0 2023-12-14T18:29:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdcbfbcb80 10 client.15786 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:29:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdcbfbcb80 10 client.15786 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:29:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdcbfbcb80 20 client.15786 populate_metadata read hostname 'smithi093' 2023-12-14T18:29:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561c9c4f3560 0x561c9c4f5970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.441 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdcbfbcb80 4 client.15786 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/1399891051 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561c9c424c80 con 0x561c9c4f3560 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.428+0000 7fcdbdbca700 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561c9c4f3560 0x561c9c4f5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.429+0000 7fcda17fa700 10 client.15786 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.429+0000 7fcdbdbca700 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561c9c4f3560 0x561c9c4f5970 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6894ec6df00a5551 server_cookie=6dcd9aeef3b8cef6 in_seq=0 out_seq=0 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.431+0000 7fcda17fa700 1 -- 172.21.15.93:0/1399891051 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561c9c424c80 con 0x561c9c4f3560 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.431+0000 7fcda17fa700 10 client.15786 handle_command_reply: tid=0 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.431+0000 7fcdcbfbcb80 1 client.15786 shutdown 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/1399891051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 msgr2=0x561c9c4e13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e13b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40a2d80 tx=0x7fcdb40a3320 comp rx=0 tx=0).stop 2023-12-14T18:29:43.442 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/1399891051 shutdown_connections 2023-12-14T18:29:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561c9c4f3560 0x561c9c4f5970 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4e40f0 0x561c9c4e13b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 0x561c9c4e3a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcda17fa700 0 client.15786 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/1399891051 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4d9370 0x561c9c4e0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/1399891051 >> 172.21.15.93:0/1399891051 conn(0x561c9c4cf890 msgr2=0x561c9c4d4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/1399891051 shutdown_connections 2023-12-14T18:29:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/1399891051 wait complete. 2023-12-14T18:29:43.443 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.432+0000 7fcdcbfbcb80 20 client.15786 trim_cache size 0 max 16384 2023-12-14T18:29:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdcbfbcb80 1 Processor -- start 2023-12-14T18:29:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdcbfbcb80 1 -- start start 2023-12-14T18:29:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 0x561c9c4eaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4d9370 0x561c9c4dbd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c4dc250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c4da0a0 con 0x561c9c4d9370 2023-12-14T18:29:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c4f5fd0 con 0x561c9c4e40f0 2023-12-14T18:29:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c4df2e0 con 0x561c9c336430 2023-12-14T18:29:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbd3c9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4d9370 0x561c9c4dbd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbe3cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c4dc250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbd3c9700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4d9370 0x561c9c4dbd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33506/0 (socket says 172.21.15.93:33506) 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbdbca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 0x561c9c4eaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbd3c9700 1 -- 172.21.15.93:0/204369062 learned_addr learned my addr 172.21.15.93:0/204369062 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/204369062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 msgr2=0x561c9c4eaaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/204369062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 0x561c9c4eaaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/204369062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4d9370 msgr2=0x561c9c4dbd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/204369062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4d9370 0x561c9c4dbd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbe3cb700 1 -- 172.21.15.93:0/204369062 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561c9c4d10d0 con 0x561c9c4e40f0 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.448+0000 7fcdbd3c9700 1 --2- 172.21.15.93:0/204369062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4d9370 0x561c9c4dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/204369062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c4dc250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40cbae0 tx=0x7fcdb40994a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f7d10ca79c0d8c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdacff9700 1 -- 172.21.15.93:0/204369062 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40b9250 con 0x561c9c4e40f0 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdacff9700 1 -- 172.21.15.93:0/204369062 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fcdb409acc0 con 0x561c9c4e40f0 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdacff9700 1 -- 172.21.15.93:0/204369062 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb40a2e20 con 0x561c9c4e40f0 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/204369062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 msgr2=0x561c9c4dc250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/204369062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c4dc250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fcdb40cbae0 tx=0x7fcdb40994a0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/204369062 shutdown_connections 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/204369062 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c4dc250 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/204369062 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c336430 0x561c9c4eaaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/204369062 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c4d9370 0x561c9c4dbd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.449+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/204369062 >> 172.21.15.93:0/204369062 conn(0x561c9c4cf890 msgr2=0x561c9c4e3690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/204369062 shutdown_connections 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/204369062 wait complete. 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 -- start start 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 Processor -- start 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c336430 0x561c9c5403c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 0x561c9c540900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c540e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c53b4e0 con 0x561c9c336430 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c53b650 con 0x561c9c4e40f0 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561c9c53b810 con 0x561c9c4d9370 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdcbfbcb80 10 client.0 _ll_register_callbacks cb 0x7fcdb0011a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdbdbca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c336430 0x561c9c5403c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdbd3c9700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 0x561c9c540900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdbe3cb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c540e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdbdbca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c336430 0x561c9c5403c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:33514/0 (socket says 172.21.15.93:33514) 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.450+0000 7fcdbdbca700 1 -- 172.21.15.93:0/2493615310 learned_addr learned my addr 172.21.15.93:0/2493615310 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdbdbca700 1 -- 172.21.15.93:0/2493615310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 msgr2=0x561c9c540900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdbdbca700 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 0x561c9c540900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdbdbca700 1 -- 172.21.15.93:0/2493615310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 msgr2=0x561c9c540e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdbdbca700 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c540e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdbdbca700 1 -- 172.21.15.93:0/2493615310 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcda8009c00 con 0x561c9c336430 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdbe3cb700 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c540e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdbdbca700 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c336430 0x561c9c5403c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x561c9c4f61e0 tx=0x7fcdb00047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7736c9207ec3876c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcda17fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:29:43.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcda17fa700 1 -- 172.21.15.93:0/2493615310 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdb0022040 con 0x561c9c336430 2023-12-14T18:29:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2493615310 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561c9c4d9200 con 0x561c9c336430 2023-12-14T18:29:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcda17fa700 1 -- 172.21.15.93:0/2493615310 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fcdb0031070 con 0x561c9c336430 2023-12-14T18:29:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcda17fa700 1 -- 172.21.15.93:0/2493615310 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcdb002be40 con 0x561c9c336430 2023-12-14T18:29:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdcbfbcb80 10 client.15792 fetch_fsmap learned FSMap version 28 2023-12-14T18:29:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2493615310 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x561c9c3f2110 con 0x561c9c336430 2023-12-14T18:29:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.451+0000 7fcda17fa700 1 -- 172.21.15.93:0/2493615310 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561c9c4d9200 con 0x561c9c336430 2023-12-14T18:29:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcda17fa700 1 -- 172.21.15.93:0/2493615310 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7fcdb00365f0 con 0x561c9c336430 2023-12-14T18:29:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcdcbfbcb80 10 client.15792 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:29:43.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcdcbfbcb80 10 client.15792 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:29:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcdcbfbcb80 20 client.15792 populate_metadata read hostname 'smithi093' 2023-12-14T18:29:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561c9c54c470 0x561c9c54e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcdcbfbcb80 4 client.15792 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:29:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2493615310 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x7fcdb409ef20 con 0x561c9c54c470 2023-12-14T18:29:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcdbd3c9700 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561c9c54c470 0x561c9c54e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcda17fa700 10 client.15792 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:43.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:43.452+0000 7fcdbd3c9700 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561c9c54c470 0x561c9c54e880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b2b1a2e2918e3a server_cookie=9cfcdd204641f978 in_seq=0 out_seq=0 2023-12-14T18:29:43.820 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:43 smithi093 ceph-mon[42070]: pgmap v100: 129 pgs: 129 active+clean; 154 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.23k op/s 2023-12-14T18:29:43.845 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:43 smithi097 ceph-mon[43486]: pgmap v100: 129 pgs: 129 active+clean; 154 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.23k op/s 2023-12-14T18:29:43.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:43 smithi160 ceph-mon[43840]: pgmap v100: 129 pgs: 129 active+clean; 154 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.23k op/s 2023-12-14T18:29:44.889 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcda17fa700 1 -- 172.21.15.93:0/2493615310 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x7fcdb409ef20 con 0x561c9c54c470 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcda17fa700 10 client.15792 handle_command_reply: tid=0 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 client.15792 shutdown 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2493615310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c336430 msgr2=0x561c9c5403c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c336430 0x561c9c5403c0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x561c9c4f61e0 tx=0x7fcdb00047c0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2493615310 shutdown_connections 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561c9c54c470 0x561c9c54e880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561c9c336430 0x561c9c5403c0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561c9c4e40f0 0x561c9c540e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcda17fa700 0 client.15792 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 --2- 172.21.15.93:0/2493615310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561c9c4d9370 0x561c9c540900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2493615310 >> 172.21.15.93:0/2493615310 conn(0x561c9c4cf890 msgr2=0x561c9c4d43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2493615310 shutdown_connections 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 1 -- 172.21.15.93:0/2493615310 wait complete. 2023-12-14T18:29:44.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.888+0000 7fcdcbfbcb80 20 client.15792 trim_cache size 0 max 16384 2023-12-14T18:29:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:29:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:29:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:29:44.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcda40a12c0 msgr2=0x7fcda40a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcda40a12c0 0x7fcda40a3770 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcdb8067590 tx=0x7fcdb0009260 comp rx=0 tx=0).stop 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 msgr2=0x7fcdb819ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 0x7fcdb819ad10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fcdb4007870 tx=0x7fcdb40078a0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/251217205 shutdown_connections 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcda40a12c0 0x7fcda40a3770 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fcdb8109a70 0x7fcdb819ad10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fcdb81004c0 0x7fcdb8196370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 --2- 172.21.15.93:0/251217205 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fcdb8100dd0 0x7fcdb81968b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/251217205 >> 172.21.15.93:0/251217205 conn(0x7fcdb8074960 msgr2=0x7fcdb80fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:44.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/251217205 shutdown_connections 2023-12-14T18:29:44.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:44.889+0000 7fcdbfe2e700 1 -- 172.21.15.93:0/251217205 wait complete. 2023-12-14T18:29:45.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:45 smithi093 ceph-mon[42070]: pgmap v101: 129 pgs: 129 active+clean; 154 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.23k op/s 2023-12-14T18:29:45.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:45 smithi097 ceph-mon[43486]: pgmap v101: 129 pgs: 129 active+clean; 154 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.23k op/s 2023-12-14T18:29:45.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:45 smithi160 ceph-mon[43840]: pgmap v101: 129 pgs: 129 active+clean; 154 MiB data, 600 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 1.23k op/s 2023-12-14T18:29:47.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:47 smithi093 ceph-mon[42070]: pgmap v102: 129 pgs: 129 active+clean; 166 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.5 MiB/s wr, 1.63k op/s 2023-12-14T18:29:47.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:47 smithi097 ceph-mon[43486]: pgmap v102: 129 pgs: 129 active+clean; 166 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.5 MiB/s wr, 1.63k op/s 2023-12-14T18:29:47.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:47 smithi160 ceph-mon[43840]: pgmap v102: 129 pgs: 129 active+clean; 166 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.5 MiB/s wr, 1.63k op/s 2023-12-14T18:29:49.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:49 smithi093 ceph-mon[42070]: pgmap v103: 129 pgs: 129 active+clean; 166 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 436 op/s 2023-12-14T18:29:49.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:49 smithi097 ceph-mon[43486]: pgmap v103: 129 pgs: 129 active+clean; 166 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 436 op/s 2023-12-14T18:29:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:49 smithi160 ceph-mon[43840]: pgmap v103: 129 pgs: 129 active+clean; 166 MiB data, 642 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 MiB/s rd, 2.3 MiB/s wr, 436 op/s 2023-12-14T18:29:51.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:51 smithi093 ceph-mon[42070]: pgmap v104: 129 pgs: 129 active+clean; 161 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 711 op/s 2023-12-14T18:29:51.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:51 smithi097 ceph-mon[43486]: pgmap v104: 129 pgs: 129 active+clean; 161 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 711 op/s 2023-12-14T18:29:51.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:51 smithi160 ceph-mon[43840]: pgmap v104: 129 pgs: 129 active+clean; 161 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.4 MiB/s wr, 711 op/s 2023-12-14T18:29:53.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:53 smithi093 ceph-mon[42070]: pgmap v105: 129 pgs: 129 active+clean; 161 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 702 op/s 2023-12-14T18:29:53.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:53 smithi097 ceph-mon[43486]: pgmap v105: 129 pgs: 129 active+clean; 161 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 702 op/s 2023-12-14T18:29:53.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:53 smithi160 ceph-mon[43840]: pgmap v105: 129 pgs: 129 active+clean; 161 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 702 op/s 2023-12-14T18:29:55.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:55 smithi093 ceph-mon[42070]: pgmap v106: 129 pgs: 129 active+clean; 161 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 702 op/s 2023-12-14T18:29:55.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:55 smithi097 ceph-mon[43486]: pgmap v106: 129 pgs: 129 active+clean; 161 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 702 op/s 2023-12-14T18:29:55.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:55 smithi160 ceph-mon[43840]: pgmap v106: 129 pgs: 129 active+clean; 161 MiB data, 683 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.2 MiB/s wr, 702 op/s 2023-12-14T18:29:57.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:57 smithi093 ceph-mon[42070]: pgmap v107: 129 pgs: 129 active+clean; 185 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 802 op/s 2023-12-14T18:29:57.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:57 smithi097 ceph-mon[43486]: pgmap v107: 129 pgs: 129 active+clean; 185 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 802 op/s 2023-12-14T18:29:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:57 smithi160 ceph-mon[43840]: pgmap v107: 129 pgs: 129 active+clean; 185 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 MiB/s rd, 4.2 MiB/s wr, 802 op/s 2023-12-14T18:29:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.005+0000 7f6b64e9e700 1 -- 172.21.15.93:0/3600310871 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101440 msgr2=0x7f6b60101810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.005+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/3600310871 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101440 0x7f6b60101810 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009990 tx=0x7f6b50009ca0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.007 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.005+0000 7f6b64e9e700 1 -- 172.21.15.93:0/3600310871 shutdown_connections 2023-12-14T18:29:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.005+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/3600310871 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101440 0x7f6b60101810 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.005+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/3600310871 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6010a810 0x7f6b6010cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.005+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/3600310871 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b60101d50 0x7f6b6010a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.005+0000 7f6b64e9e700 1 -- 172.21.15.93:0/3600310871 >> 172.21.15.93:0/3600310871 conn(0x7f6b600763a0 msgr2=0x7f6b600767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.005+0000 7f6b64e9e700 1 -- 172.21.15.93:0/3600310871 shutdown_connections 2023-12-14T18:29:58.008 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.005+0000 7f6b64e9e700 1 -- 172.21.15.93:0/3600310871 wait complete. 2023-12-14T18:29:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b64e9e700 1 Processor -- start 2023-12-14T18:29:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b64e9e700 1 -- start start 2023-12-14T18:29:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b64e9e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b60101440 0x7f6b60196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b64e9e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101d50 0x7f6b60196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b64e9e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6010a810 0x7f6b6019acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.010 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b64e9e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019b300 con 0x7f6b60101d50 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b64e9e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019b470 con 0x7f6b6010a810 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b64e9e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b6019b630 con 0x7f6b60101440 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b5dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101d50 0x7f6b60196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b5dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101d50 0x7f6b60196860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45626/0 (socket says 172.21.15.93:45626) 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/2995736385 learned_addr learned my addr 172.21.15.93:0/2995736385 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b60101440 0x7f6b60196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.006+0000 7f6b5ed9d700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6010a810 0x7f6b6019acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/2995736385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b60101440 msgr2=0x7f6b60196320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b60101440 0x7f6b60196320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/2995736385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6010a810 msgr2=0x7f6b6019acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6010a810 0x7f6b6019acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/2995736385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b500095f0 con 0x7f6b60101d50 2023-12-14T18:29:58.011 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101d50 0x7f6b60196860 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e9a0 tx=0x7f6b5400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b4f7fe700 1 -- 172.21.15.93:0/2995736385 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54009790 con 0x7f6b60101d50 2023-12-14T18:29:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b64e9e700 1 -- 172.21.15.93:0/2995736385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b60110290 con 0x7f6b60101d50 2023-12-14T18:29:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b4f7fe700 1 -- 172.21.15.93:0/2995736385 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b54010430 con 0x7f6b60101d50 2023-12-14T18:29:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b64e9e700 1 -- 172.21.15.93:0/2995736385 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b60110100 con 0x7f6b60101d50 2023-12-14T18:29:58.012 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.007+0000 7f6b4f7fe700 1 -- 172.21.15.93:0/2995736385 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b54018680 con 0x7f6b60101d50 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.009+0000 7f6b4f7fe700 1 -- 172.21.15.93:0/2995736385 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6b5400ca10 con 0x7f6b60101d50 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.010+0000 7f6b4f7fe700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6b480a12c0 0x7f6b480a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.010+0000 7f6b4f7fe700 1 -- 172.21.15.93:0/2995736385 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f6b54014070 con 0x7f6b60101d50 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.010+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6b480a12c0 0x7f6b480a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6b480a12c0 0x7f6b480a3770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000b2e0 tx=0x7f6b50005fd0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b7102cb80 1 Processor -- start 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b7102cb80 1 -- start start 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81d77430 0x555d81f19e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81f1a370 0x555d81f1a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.018 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f274d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81e04580 con 0x555d81f1a370 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81e73810 con 0x555d81d77430 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81e6cca0 con 0x555d81f250f0 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b5dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81f1a370 0x555d81f1a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b5ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f274d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81d77430 0x555d81f19e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b5dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81f1a370 0x555d81f1a7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45640/0 (socket says 172.21.15.93:45640) 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.011+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/2624917668 learned_addr learned my addr 172.21.15.93:0/2624917668 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b5ed9d700 1 -- 172.21.15.93:0/2624917668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81d77430 msgr2=0x555d81f19e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b5ed9d700 1 --2- 172.21.15.93:0/2624917668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81d77430 0x555d81f19e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b5ed9d700 1 -- 172.21.15.93:0/2624917668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81f1a370 msgr2=0x555d81f1a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b5ed9d700 1 --2- 172.21.15.93:0/2624917668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81f1a370 0x555d81f1a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b5ed9d700 1 -- 172.21.15.93:0/2624917668 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d81cee350 con 0x555d81f250f0 2023-12-14T18:29:58.019 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/2624917668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81f1a370 0x555d81f1a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b5ed9d700 1 --2- 172.21.15.93:0/2624917668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f274d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b58011550 tx=0x7f6b58009920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5feb26840eb058ca server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b4cff9700 1 -- 172.21.15.93:0/2624917668 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5800e040 con 0x555d81f250f0 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b4cff9700 1 -- 172.21.15.93:0/2624917668 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b5800f040 con 0x555d81f250f0 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.012+0000 7f6b4cff9700 1 -- 172.21.15.93:0/2624917668 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b58004320 con 0x555d81f250f0 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2624917668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 msgr2=0x555d81f274d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/2624917668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f274d0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f6b58011550 tx=0x7f6b58009920 comp rx=0 tx=0).stop 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2624917668 shutdown_connections 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/2624917668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f274d0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/2624917668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81d77430 0x555d81f19e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/2624917668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81f1a370 0x555d81f1a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2624917668 >> 172.21.15.93:0/2624917668 conn(0x555d81f10890 msgr2=0x555d81f16200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2624917668 shutdown_connections 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2624917668 wait complete. 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 -- start start 2023-12-14T18:29:58.020 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.013+0000 7f6b7102cb80 1 Processor -- start 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f24930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f1a370 0x555d81f21c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f250f0 0x555d81f22190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81e08f30 con 0x555d81d77430 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81e08bd0 con 0x555d81f250f0 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81cf0640 con 0x555d81f1a370 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b7102cb80 10 client.0 _ll_register_callbacks cb 0x555d81e2c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f24930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f1a370 0x555d81f21c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f24930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45642/0 (socket says 172.21.15.93:45642) 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5e59c700 1 -- 172.21.15.93:0/4193856726 learned_addr learned my addr 172.21.15.93:0/4193856726 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5ed9d700 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f250f0 0x555d81f22190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/4193856726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f250f0 msgr2=0x555d81f22190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.021 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f250f0 0x555d81f22190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/4193856726 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 msgr2=0x555d81f24930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f24930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/4193856726 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b58011230 con 0x555d81f1a370 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f24930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f1a370 0x555d81f21c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b54099e60 tx=0x7f6b5409df80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71f62179e44aa40 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.014+0000 7f6b45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b45ffb700 1 -- 172.21.15.93:0/4193856726 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5409a400 con 0x555d81f1a370 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b7102cb80 1 -- 172.21.15.93:0/4193856726 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555d81f0dea0 con 0x555d81f1a370 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b45ffb700 1 -- 172.21.15.93:0/4193856726 <== mon.1 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f6b540995c0 con 0x555d81f1a370 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b45ffb700 1 -- 172.21.15.93:0/4193856726 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b540adbd0 con 0x555d81f1a370 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b7102cb80 10 client.25420 fetch_fsmap learned FSMap version 28 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b7102cb80 1 -- 172.21.15.93:0/4193856726 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555d81e33110 con 0x555d81f1a370 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b45ffb700 1 -- 172.21.15.93:0/4193856726 <== mon.1 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555d81f0dea0 con 0x555d81f1a370 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b45ffb700 1 -- 172.21.15.93:0/4193856726 <== mon.1 v2:172.21.15.160:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6b540b7fa0 con 0x555d81f1a370 2023-12-14T18:29:58.022 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b7102cb80 10 client.25420 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:29:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b7102cb80 10 client.25420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:29:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b7102cb80 20 client.25420 populate_metadata read hostname 'smithi093' 2023-12-14T18:29:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555d81f34b30 0x555d81f36f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b7102cb80 4 client.25420 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:29:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b7102cb80 1 -- 172.21.15.93:0/4193856726 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555d81e65c80 con 0x555d81f34b30 2023-12-14T18:29:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.015+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555d81f34b30 0x555d81f36f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.016+0000 7f6b45ffb700 10 client.25420 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:58.023 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.016+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555d81f34b30 0x555d81f36f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ea430267019d2934 server_cookie=2468ee1c382f5ab in_seq=0 out_seq=0 2023-12-14T18:29:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b45ffb700 1 -- 172.21.15.93:0/4193856726 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555d81e65c80 con 0x555d81f34b30 2023-12-14T18:29:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b45ffb700 10 client.25420 handle_command_reply: tid=0 2023-12-14T18:29:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 client.25420 shutdown 2023-12-14T18:29:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 -- 172.21.15.93:0/4193856726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f1a370 msgr2=0x555d81f21c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f1a370 0x555d81f21c50 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6b54099e60 tx=0x7f6b5409df80 comp rx=0 tx=0).stop 2023-12-14T18:29:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 -- 172.21.15.93:0/4193856726 shutdown_connections 2023-12-14T18:29:58.024 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555d81f34b30 0x555d81f36f40 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f24930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f250f0 0x555d81f22190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b45ffb700 0 client.25420 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/4193856726 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f1a370 0x555d81f21c50 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 -- 172.21.15.93:0/4193856726 >> 172.21.15.93:0/4193856726 conn(0x555d81f10890 msgr2=0x555d81f15d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 -- 172.21.15.93:0/4193856726 shutdown_connections 2023-12-14T18:29:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 1 -- 172.21.15.93:0/4193856726 wait complete. 2023-12-14T18:29:58.025 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.018+0000 7f6b7102cb80 20 client.25420 trim_cache size 0 max 16384 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b7102cb80 1 Processor -- start 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b7102cb80 1 -- start start 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f2bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f24460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f1eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81f23280 con 0x555d81d77430 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81f227a0 con 0x555d81f1a370 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81f22c90 con 0x555d81f250f0 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f2bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f24460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5ed9d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f1eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.037 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5dd9b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f24460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:53246/0 (socket says 172.21.15.93:53246) 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/2909748767 learned_addr learned my addr 172.21.15.93:0/2909748767 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/2909748767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f2bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45646/0 (socket says 172.21.15.93:45646) 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/2909748767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 msgr2=0x555d81f1eec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/2909748767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f1eec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/2909748767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 msgr2=0x555d81f2bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/2909748767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f2bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.033+0000 7f6b5dd9b700 1 -- 172.21.15.93:0/2909748767 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555d81f120d0 con 0x555d81f1a370 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.034+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/2909748767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f24460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b540cab50 tx=0x7f6b54099ca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bfe40066241f6327 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.034+0000 7f6b4d7fa700 1 -- 172.21.15.93:0/2909748767 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5409a400 con 0x555d81f1a370 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.034+0000 7f6b4d7fa700 1 -- 172.21.15.93:0/2909748767 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b54096210 con 0x555d81f1a370 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.034+0000 7f6b4d7fa700 1 -- 172.21.15.93:0/2909748767 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b540993c0 con 0x555d81f1a370 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2909748767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 msgr2=0x555d81f24460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/2909748767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f24460 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f6b540cab50 tx=0x7f6b54099ca0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2909748767 shutdown_connections 2023-12-14T18:29:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/2909748767 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f2bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/2909748767 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f1eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/2909748767 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f24460 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2909748767 >> 172.21.15.93:0/2909748767 conn(0x555d81f10890 msgr2=0x555d81f15cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2909748767 shutdown_connections 2023-12-14T18:29:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 -- 172.21.15.93:0/2909748767 wait complete. 2023-12-14T18:29:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 -- start start 2023-12-14T18:29:58.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 Processor -- start 2023-12-14T18:29:58.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f80670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f2b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f2b940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81f2bf90 con 0x555d81d77430 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81f2c100 con 0x555d81f1a370 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555d81f2c2c0 con 0x555d81f250f0 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b7102cb80 10 client.0 _ll_register_callbacks cb 0x7f6b54099ed0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f80670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b5e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f80670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45662/0 (socket says 172.21.15.93:45662) 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.035+0000 7f6b5e59c700 1 -- 172.21.15.93:0/36853310 learned_addr learned my addr 172.21.15.93:0/36853310 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f2b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b5ed9d700 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f2b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b5e59c700 1 -- 172.21.15.93:0/36853310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 msgr2=0x555d81f2b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.042 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f2b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b5e59c700 1 -- 172.21.15.93:0/36853310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 msgr2=0x555d81f2b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f2b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b5e59c700 1 -- 172.21.15.93:0/36853310 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555d81e33110 con 0x555d81d77430 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f2b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b5e59c700 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f80670 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009990 tx=0x7f6b50009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c1b253bc3e7384c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b45ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b45ffb700 1 -- 172.21.15.93:0/36853310 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b5002b040 con 0x555d81d77430 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b45ffb700 1 -- 172.21.15.93:0/36853310 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f6b50035ac0 con 0x555d81d77430 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b7102cb80 1 -- 172.21.15.93:0/36853310 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f6b5800f040 con 0x555d81d77430 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b45ffb700 1 -- 172.21.15.93:0/36853310 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b50035c80 con 0x555d81d77430 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b7102cb80 10 client.15810 fetch_fsmap learned FSMap version 28 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.036+0000 7f6b7102cb80 1 -- 172.21.15.93:0/36853310 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x555d81c9ab00 con 0x555d81d77430 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b45ffb700 1 -- 172.21.15.93:0/36853310 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6b5800f040 con 0x555d81d77430 2023-12-14T18:29:58.043 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b45ffb700 1 -- 172.21.15.93:0/36853310 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f6b5003e470 con 0x555d81d77430 2023-12-14T18:29:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b7102cb80 10 client.15810 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:29:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b7102cb80 10 client.15810 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:29:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b7102cb80 20 client.15810 populate_metadata read hostname 'smithi093' 2023-12-14T18:29:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555d81f91bf0 0x555d81f94000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:29:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b7102cb80 4 client.15810 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:29:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b7102cb80 1 -- 172.21.15.93:0/36853310 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555d81e659a0 con 0x555d81f91bf0 2023-12-14T18:29:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555d81f91bf0 0x555d81f94000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:29:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b45ffb700 10 client.15810 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:58.044 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:58.037+0000 7f6b5dd9b700 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555d81f91bf0 0x555d81f94000 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4fe2af0c175380b server_cookie=fb1524a6e5e1e0f5 in_seq=0 out_seq=0 2023-12-14T18:29:59.038 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b45ffb700 1 -- 172.21.15.93:0/36853310 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555d81e659a0 con 0x555d81f91bf0 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b45ffb700 10 client.15810 handle_command_reply: tid=0 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 client.15810 shutdown 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 -- 172.21.15.93:0/36853310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 msgr2=0x555d81f80670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f80670 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6b50009990 tx=0x7f6b50009ca0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 -- 172.21.15.93:0/36853310 shutdown_connections 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555d81f91bf0 0x555d81f94000 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555d81d77430 0x555d81f80670 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555d81f250f0 0x555d81f2b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b45ffb700 0 client.15810 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 --2- 172.21.15.93:0/36853310 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555d81f1a370 0x555d81f2b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 -- 172.21.15.93:0/36853310 >> 172.21.15.93:0/36853310 conn(0x555d81f10890 msgr2=0x555d81f304c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 -- 172.21.15.93:0/36853310 shutdown_connections 2023-12-14T18:29:59.039 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 1 -- 172.21.15.93:0/36853310 wait complete. 2023-12-14T18:29:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.036+0000 7f6b7102cb80 20 client.15810 trim_cache size 0 max 16384 2023-12-14T18:29:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:29:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:29:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:29:59.040 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 -- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6b480a12c0 msgr2=0x7f6b480a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6b480a12c0 0x7f6b480a3770 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b5000b2e0 tx=0x7f6b50005fd0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 -- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101d50 msgr2=0x7f6b60196860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101d50 0x7f6b60196860 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f6b5400e9a0 tx=0x7f6b5400ecb0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 -- 172.21.15.93:0/2995736385 shutdown_connections 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6b6010a810 0x7f6b6019acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6b60101440 0x7f6b60196320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6b480a12c0 0x7f6b480a3770 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 --2- 172.21.15.93:0/2995736385 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6b60101d50 0x7f6b60196860 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 -- 172.21.15.93:0/2995736385 >> 172.21.15.93:0/2995736385 conn(0x7f6b600763a0 msgr2=0x7f6b600fd950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 -- 172.21.15.93:0/2995736385 shutdown_connections 2023-12-14T18:29:59.041 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:29:59.037+0000 7f6b64e9e700 1 -- 172.21.15.93:0/2995736385 wait complete. 2023-12-14T18:29:59.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:29:59 smithi097 ceph-mon[43486]: pgmap v108: 129 pgs: 129 active+clean; 185 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 375 op/s 2023-12-14T18:29:59.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:29:59 smithi160 ceph-mon[43840]: pgmap v108: 129 pgs: 129 active+clean; 185 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 375 op/s 2023-12-14T18:30:00.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:29:59 smithi093 ceph-mon[42070]: pgmap v108: 129 pgs: 129 active+clean; 185 MiB data, 633 MiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.0 MiB/s wr, 375 op/s 2023-12-14T18:30:00.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:00 smithi097 ceph-mon[43486]: overall HEALTH_OK 2023-12-14T18:30:00.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:00 smithi160 ceph-mon[43840]: overall HEALTH_OK 2023-12-14T18:30:01.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:00 smithi093 ceph-mon[42070]: overall HEALTH_OK 2023-12-14T18:30:01.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:01 smithi097 ceph-mon[43486]: pgmap v109: 129 pgs: 129 active+clean; 189 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 895 op/s 2023-12-14T18:30:01.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:01 smithi160 ceph-mon[43840]: pgmap v109: 129 pgs: 129 active+clean; 189 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 895 op/s 2023-12-14T18:30:02.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:01 smithi093 ceph-mon[42070]: pgmap v109: 129 pgs: 129 active+clean; 189 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 MiB/s rd, 4.9 MiB/s wr, 895 op/s 2023-12-14T18:30:03.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:03 smithi097 ceph-mon[43486]: pgmap v110: 129 pgs: 129 active+clean; 189 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 619 op/s 2023-12-14T18:30:03.995 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:03 smithi160 ceph-mon[43840]: pgmap v110: 129 pgs: 129 active+clean; 189 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 619 op/s 2023-12-14T18:30:04.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:03 smithi093 ceph-mon[42070]: pgmap v110: 129 pgs: 129 active+clean; 189 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 619 op/s 2023-12-14T18:30:05.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:05 smithi097 ceph-mon[43486]: pgmap v111: 129 pgs: 129 active+clean; 189 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 619 op/s 2023-12-14T18:30:05.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:05 smithi160 ceph-mon[43840]: pgmap v111: 129 pgs: 129 active+clean; 189 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 619 op/s 2023-12-14T18:30:06.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:05 smithi093 ceph-mon[42070]: pgmap v111: 129 pgs: 129 active+clean; 189 MiB data, 635 MiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.8 MiB/s wr, 619 op/s 2023-12-14T18:30:07.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:07 smithi097 ceph-mon[43486]: pgmap v112: 129 pgs: 129 active+clean; 221 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 697 op/s 2023-12-14T18:30:07.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:07 smithi160 ceph-mon[43840]: pgmap v112: 129 pgs: 129 active+clean; 221 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 697 op/s 2023-12-14T18:30:08.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:07 smithi093 ceph-mon[42070]: pgmap v112: 129 pgs: 129 active+clean; 221 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 MiB/s rd, 7.0 MiB/s wr, 697 op/s 2023-12-14T18:30:09.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:09 smithi160 ceph-mon[43840]: pgmap v113: 129 pgs: 129 active+clean; 221 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 597 op/s 2023-12-14T18:30:10.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:09 smithi093 ceph-mon[42070]: pgmap v113: 129 pgs: 129 active+clean; 221 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 597 op/s 2023-12-14T18:30:10.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:09 smithi097 ceph-mon[43486]: pgmap v113: 129 pgs: 129 active+clean; 221 MiB data, 689 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 5.1 MiB/s wr, 597 op/s 2023-12-14T18:30:11.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:11 smithi160 ceph-mon[43840]: pgmap v114: 129 pgs: 129 active+clean; 276 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.7 MiB/s wr, 622 op/s 2023-12-14T18:30:12.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:11 smithi093 ceph-mon[42070]: pgmap v114: 129 pgs: 129 active+clean; 276 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.7 MiB/s wr, 622 op/s 2023-12-14T18:30:12.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:11 smithi097 ceph-mon[43486]: pgmap v114: 129 pgs: 129 active+clean; 276 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 9.7 MiB/s wr, 622 op/s 2023-12-14T18:30:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.146+0000 7f5709b96700 1 -- 172.21.15.93:0/1033466591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57040ffd60 msgr2=0x7f5704103d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.146+0000 7f5709b96700 1 --2- 172.21.15.93:0/1033466591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57040ffd60 0x7f5704103d20 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0099c0 tx=0x7f56ec009cd0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.146+0000 7f5709b96700 1 -- 172.21.15.93:0/1033466591 shutdown_connections 2023-12-14T18:30:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.146+0000 7f5709b96700 1 --2- 172.21.15.93:0/1033466591 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f5704104400 0x7f570410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.146+0000 7f5709b96700 1 --2- 172.21.15.93:0/1033466591 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57040ff3c0 0x7f57040ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.146+0000 7f5709b96700 1 --2- 172.21.15.93:0/1033466591 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f57040ffd60 0x7f5704103d20 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.146+0000 7f5709b96700 1 -- 172.21.15.93:0/1033466591 >> 172.21.15.93:0/1033466591 conn(0x7f57040763a0 msgr2=0x7f57040767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:12.151 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.146+0000 7f5709b96700 1 -- 172.21.15.93:0/1033466591 shutdown_connections 2023-12-14T18:30:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.146+0000 7f5709b96700 1 -- 172.21.15.93:0/1033466591 wait complete. 2023-12-14T18:30:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.147+0000 7f5709b96700 1 Processor -- start 2023-12-14T18:30:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.147+0000 7f5709b96700 1 -- start start 2023-12-14T18:30:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.147+0000 7f5709b96700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57040ff3c0 0x7f570410c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.147+0000 7f5709b96700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57040ffd60 0x7f570410ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.147+0000 7f5709b96700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5704104400 0x7f570410d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.147+0000 7f5709b96700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57041a4840 con 0x7f5704104400 2023-12-14T18:30:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.147+0000 7f5709b96700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57041a49b0 con 0x7f57040ff3c0 2023-12-14T18:30:12.152 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.147+0000 7f5709b96700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57041a4b70 con 0x7f57040ffd60 2023-12-14T18:30:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5703fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5704104400 0x7f570410d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5702ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57040ffd60 0x7f570410ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5703fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5704104400 0x7f570410d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55258/0 (socket says 172.21.15.93:55258) 2023-12-14T18:30:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f57037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57040ff3c0 0x7f570410c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5703fff700 1 -- 172.21.15.93:0/2584586047 learned_addr learned my addr 172.21.15.93:0/2584586047 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5702ffd700 1 -- 172.21.15.93:0/2584586047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57040ff3c0 msgr2=0x7f570410c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:12.153 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5702ffd700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57040ff3c0 0x7f570410c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5702ffd700 1 -- 172.21.15.93:0/2584586047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5704104400 msgr2=0x7f570410d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5702ffd700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5704104400 0x7f570410d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5702ffd700 1 -- 172.21.15.93:0/2584586047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56ec0095f0 con 0x7f57040ffd60 2023-12-14T18:30:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5702ffd700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57040ffd60 0x7f570410ce20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009fd0 tx=0x7f56ec00f690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5700ff9700 1 -- 172.21.15.93:0/2584586047 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec01d070 con 0x7f57040ffd60 2023-12-14T18:30:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5700ff9700 1 -- 172.21.15.93:0/2584586047 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56ec022470 con 0x7f57040ffd60 2023-12-14T18:30:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5709b96700 1 -- 172.21.15.93:0/2584586047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5704110330 con 0x7f57040ffd60 2023-12-14T18:30:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5700ff9700 1 -- 172.21.15.93:0/2584586047 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec017650 con 0x7f57040ffd60 2023-12-14T18:30:12.154 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.148+0000 7f5709b96700 1 -- 172.21.15.93:0/2584586047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f57041101a0 con 0x7f57040ffd60 2023-12-14T18:30:12.162 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.151+0000 7f5700ff9700 1 -- 172.21.15.93:0/2584586047 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f56ec0225e0 con 0x7f57040ffd60 2023-12-14T18:30:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.151+0000 7f5700ff9700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56f00a12c0 0x7f56f00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.151+0000 7f57037fe700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56f00a12c0 0x7f56f00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.151+0000 7f5700ff9700 1 -- 172.21.15.93:0/2584586047 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f56ec0cdb70 con 0x7f57040ffd60 2023-12-14T18:30:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56f00a12c0 0x7f56f00a3770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f56f400f540 tx=0x7f56f400b040 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5715d24b80 1 Processor -- start 2023-12-14T18:30:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5715d24b80 1 -- start start 2023-12-14T18:30:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d081430 0x563f7d223e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d224370 0x563f7d2247e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.163 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d2314d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5715d24b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d10e580 con 0x563f7d081430 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5715d24b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d17d810 con 0x563f7d224370 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5715d24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d176ca0 con 0x563f7d22f0f0 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d081430 0x563f7d223e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d081430 0x563f7d223e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55270/0 (socket says 172.21.15.93:55270) 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 -- 172.21.15.93:0/1877537668 learned_addr learned my addr 172.21.15.93:0/1877537668 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5702ffd700 1 --2- 172.21.15.93:0/1877537668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d224370 0x563f7d2247e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f5703fff700 1 --2- 172.21.15.93:0/1877537668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d2314d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 -- 172.21.15.93:0/1877537668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 msgr2=0x563f7d2314d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 --2- 172.21.15.93:0/1877537668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d2314d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.164 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 -- 172.21.15.93:0/1877537668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d224370 msgr2=0x563f7d2247e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 --2- 172.21.15.93:0/1877537668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d224370 0x563f7d2247e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 -- 172.21.15.93:0/1877537668 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f7cff8350 con 0x563f7d081430 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.152+0000 7f57037fe700 1 --2- 172.21.15.93:0/1877537668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d081430 0x563f7d223e30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f56f4003d20 tx=0x7f56f4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33077ccf9508fa1f server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f56f9ffb700 1 -- 172.21.15.93:0/1877537668 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f400e040 con 0x563f7d081430 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f56f9ffb700 1 -- 172.21.15.93:0/1877537668 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56f401b070 con 0x563f7d081430 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f56f9ffb700 1 -- 172.21.15.93:0/1877537668 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4008e40 con 0x563f7d081430 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f5715d24b80 1 -- 172.21.15.93:0/1877537668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d081430 msgr2=0x563f7d223e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f5715d24b80 1 --2- 172.21.15.93:0/1877537668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d081430 0x563f7d223e30 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f56f4003d20 tx=0x7f56f4014000 comp rx=0 tx=0).stop 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f5715d24b80 1 -- 172.21.15.93:0/1877537668 shutdown_connections 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f5715d24b80 1 --2- 172.21.15.93:0/1877537668 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d081430 0x563f7d223e30 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.165 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f5715d24b80 1 --2- 172.21.15.93:0/1877537668 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d2314d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f5715d24b80 1 --2- 172.21.15.93:0/1877537668 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d224370 0x563f7d2247e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f5715d24b80 1 -- 172.21.15.93:0/1877537668 >> 172.21.15.93:0/1877537668 conn(0x563f7d21a890 msgr2=0x563f7d220200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f5715d24b80 1 -- 172.21.15.93:0/1877537668 shutdown_connections 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.153+0000 7f5715d24b80 1 -- 172.21.15.93:0/1877537668 wait complete. 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5715d24b80 1 -- start start 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5715d24b80 1 Processor -- start 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d22ea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d22bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d22c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5715d24b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d112f30 con 0x563f7d224370 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5715d24b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d112bd0 con 0x563f7d081430 2023-12-14T18:30:12.166 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5715d24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7cffa640 con 0x563f7d22f0f0 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5715d24b80 10 client.0 _ll_register_callbacks cb 0x563f7d1368b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5702ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d22bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5702ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d22bd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55276/0 (socket says 172.21.15.93:55276) 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f57037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d22ea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5702ffd700 1 -- 172.21.15.93:0/4260557776 learned_addr learned my addr 172.21.15.93:0/4260557776 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.154+0000 7f5703fff700 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d22c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5702ffd700 1 -- 172.21.15.93:0/4260557776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 msgr2=0x563f7d22c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5702ffd700 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d22c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5702ffd700 1 -- 172.21.15.93:0/4260557776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 msgr2=0x563f7d22ea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5702ffd700 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d22ea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.167 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5702ffd700 1 -- 172.21.15.93:0/4260557776 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56f4003a00 con 0x563f7d224370 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5702ffd700 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d22bd20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f56ec09a680 tx=0x7f56ec09aa00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95360d94ca9590ab server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f56e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f56e2ffd700 1 -- 172.21.15.93:0/4260557776 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0a2d40 con 0x563f7d224370 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f56e2ffd700 1 -- 172.21.15.93:0/4260557776 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f56ec099e70 con 0x563f7d224370 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5715d24b80 1 -- 172.21.15.93:0/4260557776 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563f7d217ea0 con 0x563f7d224370 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f56e2ffd700 1 -- 172.21.15.93:0/4260557776 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56ec09a030 con 0x563f7d224370 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5715d24b80 10 client.15822 fetch_fsmap learned FSMap version 28 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5715d24b80 1 -- 172.21.15.93:0/4260557776 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f7d13d110 con 0x563f7d224370 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f56e2ffd700 1 -- 172.21.15.93:0/4260557776 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x563f7d217ea0 con 0x563f7d224370 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f56e2ffd700 1 -- 172.21.15.93:0/4260557776 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f56ec096970 con 0x563f7d224370 2023-12-14T18:30:12.168 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5715d24b80 10 client.15822 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5715d24b80 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5715d24b80 20 client.15822 populate_metadata read hostname 'smithi093' 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5715d24b80 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563f7d23e810 0x563f7d240c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5715d24b80 4 client.15822 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.155+0000 7f5715d24b80 1 -- 172.21.15.93:0/4260557776 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563f7d16fc80 con 0x563f7d23e810 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.156+0000 7f57037fe700 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563f7d23e810 0x563f7d240c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.156+0000 7f56e2ffd700 10 client.15822 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.156+0000 7f57037fe700 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563f7d23e810 0x563f7d240c20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=84326ebf85425500 server_cookie=babd309f4a947958 in_seq=0 out_seq=0 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f56e2ffd700 1 -- 172.21.15.93:0/4260557776 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x563f7d16fc80 con 0x563f7d23e810 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f56e2ffd700 10 client.15822 handle_command_reply: tid=0 2023-12-14T18:30:12.169 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 client.15822 shutdown 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 -- 172.21.15.93:0/4260557776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 msgr2=0x563f7d22bd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d22bd20 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f56ec09a680 tx=0x7f56ec09aa00 comp rx=0 tx=0).stop 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 -- 172.21.15.93:0/4260557776 shutdown_connections 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563f7d23e810 0x563f7d240c20 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d22c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f56e2ffd700 0 client.15822 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d22ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 --2- 172.21.15.93:0/4260557776 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d22bd20 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 -- 172.21.15.93:0/4260557776 >> 172.21.15.93:0/4260557776 conn(0x563f7d21a890 msgr2=0x563f7d21fd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 -- 172.21.15.93:0/4260557776 shutdown_connections 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.159+0000 7f5715d24b80 1 -- 172.21.15.93:0/4260557776 wait complete. 2023-12-14T18:30:12.170 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.160+0000 7f5715d24b80 20 client.15822 trim_cache size 0 max 16384 2023-12-14T18:30:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.174+0000 7f5715d24b80 1 Processor -- start 2023-12-14T18:30:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.174+0000 7f5715d24b80 1 -- start start 2023-12-14T18:30:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.174+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d235cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.174+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d226fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.174+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d2274f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.174+0000 7f5715d24b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d225d30 con 0x563f7d224370 2023-12-14T18:30:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.174+0000 7f5715d24b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d231a70 con 0x563f7d081430 2023-12-14T18:30:12.182 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.174+0000 7f5715d24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d2217d0 con 0x563f7d22f0f0 2023-12-14T18:30:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.174+0000 7f5702ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d226fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f5702ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d226fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55288/0 (socket says 172.21.15.93:55288) 2023-12-14T18:30:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f5702ffd700 1 -- 172.21.15.93:0/2190844847 learned_addr learned my addr 172.21.15.93:0/2190844847 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f57037fe700 1 --2- 172.21.15.93:0/2190844847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d235cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f5703fff700 1 --2- 172.21.15.93:0/2190844847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d2274f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f5702ffd700 1 -- 172.21.15.93:0/2190844847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 msgr2=0x563f7d2274f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f5702ffd700 1 --2- 172.21.15.93:0/2190844847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d2274f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f5702ffd700 1 -- 172.21.15.93:0/2190844847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 msgr2=0x563f7d235cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:12.183 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f5702ffd700 1 --2- 172.21.15.93:0/2190844847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d235cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f5702ffd700 1 -- 172.21.15.93:0/2190844847 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563f7d13ce60 con 0x563f7d224370 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f5702ffd700 1 --2- 172.21.15.93:0/2190844847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d226fb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0036c0 tx=0x7f56ec09eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a0d4cac1615f328 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f56fa7fc700 1 -- 172.21.15.93:0/2190844847 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec099aa0 con 0x563f7d224370 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f56fa7fc700 1 -- 172.21.15.93:0/2190844847 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56ec0a2d40 con 0x563f7d224370 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.175+0000 7f56fa7fc700 1 -- 172.21.15.93:0/2190844847 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56ec0a28b0 con 0x563f7d224370 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 -- 172.21.15.93:0/2190844847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 msgr2=0x563f7d226fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 --2- 172.21.15.93:0/2190844847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d226fb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f56ec0036c0 tx=0x7f56ec09eb30 comp rx=0 tx=0).stop 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 -- 172.21.15.93:0/2190844847 shutdown_connections 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 --2- 172.21.15.93:0/2190844847 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d22f0f0 0x563f7d2274f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 --2- 172.21.15.93:0/2190844847 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d235cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.184 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 --2- 172.21.15.93:0/2190844847 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d224370 0x563f7d226fb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 -- 172.21.15.93:0/2190844847 >> 172.21.15.93:0/2190844847 conn(0x563f7d21a890 msgr2=0x563f7d23ae00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 -- 172.21.15.93:0/2190844847 shutdown_connections 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 -- 172.21.15.93:0/2190844847 wait complete. 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 -- start start 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.176+0000 7f5715d24b80 1 Processor -- start 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d233ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d224370 0x563f7d234000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5715d24b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d22f0f0 0x563f7d2916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5715d24b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d291c10 con 0x563f7d22f0f0 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5715d24b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d291d50 con 0x563f7d081430 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5715d24b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x563f7d291e90 con 0x563f7d224370 2023-12-14T18:30:12.185 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5715d24b80 10 client.0 _ll_register_callbacks cb 0x7f56ec09a4c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5703fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d22f0f0 0x563f7d2916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f57037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d233ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5702ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d224370 0x563f7d234000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f57037fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d233ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:36490/0 (socket says 172.21.15.93:36490) 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5703fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d22f0f0 0x563f7d2916d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:55290/0 (socket says 172.21.15.93:55290) 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f57037fe700 1 -- 172.21.15.93:0/397509388 learned_addr learned my addr 172.21.15.93:0/397509388 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f57037fe700 1 -- 172.21.15.93:0/397509388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d224370 msgr2=0x563f7d234000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f57037fe700 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d224370 0x563f7d234000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f57037fe700 1 -- 172.21.15.93:0/397509388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d22f0f0 msgr2=0x563f7d2916d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:12.186 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f57037fe700 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d22f0f0 0x563f7d2916d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f57037fe700 1 -- 172.21.15.93:0/397509388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x563f7d13d110 con 0x563f7d081430 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5702ffd700 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d224370 0x563f7d234000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5703fff700 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d22f0f0 0x563f7d2916d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f56e2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f57037fe700 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d233ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009a00 tx=0x7f56f4004710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba2a306d66fcd44d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f56e2ffd700 1 -- 172.21.15.93:0/397509388 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56f4032070 con 0x563f7d081430 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.177+0000 7f5715d24b80 1 -- 172.21.15.93:0/397509388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f56f401b070 con 0x563f7d081430 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f56e2ffd700 1 -- 172.21.15.93:0/397509388 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f56f4004820 con 0x563f7d081430 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f56e2ffd700 1 -- 172.21.15.93:0/397509388 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7f56f400e040 con 0x563f7d081430 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f5715d24b80 10 client.25448 fetch_fsmap learned FSMap version 28 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f5715d24b80 1 -- 172.21.15.93:0/397509388 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x563f7d123940 con 0x563f7d081430 2023-12-14T18:30:12.187 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f56e2ffd700 1 -- 172.21.15.93:0/397509388 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56f401b070 con 0x563f7d081430 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f56e2ffd700 1 -- 172.21.15.93:0/397509388 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7f56f4005690 con 0x563f7d081430 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f5715d24b80 10 client.25448 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f5715d24b80 10 client.25448 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f5715d24b80 20 client.25448 populate_metadata read hostname 'smithi093' 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f5715d24b80 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563f7d2935d0 0x563f7d2959e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f5715d24b80 4 client.25448 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f5715d24b80 1 -- 172.21.15.93:0/397509388 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x563f7d16f9a0 con 0x563f7d2935d0 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.178+0000 7f5702ffd700 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563f7d2935d0 0x563f7d2959e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.179+0000 7f56e2ffd700 10 client.25448 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:12.188 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:12.179+0000 7f5702ffd700 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563f7d2935d0 0x563f7d2959e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9a918ce6a92f59de server_cookie=89066fb4e708416b in_seq=0 out_seq=0 2023-12-14T18:30:13.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:13 smithi160 ceph-mon[43840]: pgmap v115: 129 pgs: 129 active+clean; 276 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 102 op/s 2023-12-14T18:30:14.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:13 smithi093 ceph-mon[42070]: pgmap v115: 129 pgs: 129 active+clean; 276 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 102 op/s 2023-12-14T18:30:14.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:13 smithi097 ceph-mon[43486]: pgmap v115: 129 pgs: 129 active+clean; 276 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 102 op/s 2023-12-14T18:30:15.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:15 smithi160 ceph-mon[43840]: pgmap v116: 129 pgs: 129 active+clean; 276 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 102 op/s 2023-12-14T18:30:16.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:15 smithi093 ceph-mon[42070]: pgmap v116: 129 pgs: 129 active+clean; 276 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 102 op/s 2023-12-14T18:30:16.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:15 smithi097 ceph-mon[43486]: pgmap v116: 129 pgs: 129 active+clean; 276 MiB data, 716 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 7.8 MiB/s wr, 102 op/s 2023-12-14T18:30:17.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:17 smithi160 ceph-mon[43840]: pgmap v117: 129 pgs: 129 active+clean; 332 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 121 op/s 2023-12-14T18:30:18.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:17 smithi093 ceph-mon[42070]: pgmap v117: 129 pgs: 129 active+clean; 332 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 121 op/s 2023-12-14T18:30:18.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:17 smithi097 ceph-mon[43486]: pgmap v117: 129 pgs: 129 active+clean; 332 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 13 MiB/s wr, 121 op/s 2023-12-14T18:30:19.619 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f56e2ffd700 1 -- 172.21.15.93:0/397509388 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x563f7d16f9a0 con 0x563f7d2935d0 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f56e2ffd700 10 client.25448 handle_command_reply: tid=0 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 client.25448 shutdown 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 -- 172.21.15.93:0/397509388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 msgr2=0x563f7d233ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d233ac0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f56f4009a00 tx=0x7f56f4004710 comp rx=0 tx=0).stop 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 -- 172.21.15.93:0/397509388 shutdown_connections 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x563f7d2935d0 0x563f7d2959e0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x563f7d22f0f0 0x563f7d2916d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x563f7d081430 0x563f7d233ac0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 --2- 172.21.15.93:0/397509388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x563f7d224370 0x563f7d234000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f56e2ffd700 0 client.25448 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 -- 172.21.15.93:0/397509388 >> 172.21.15.93:0/397509388 conn(0x563f7d21a890 msgr2=0x563f7d21fd30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 -- 172.21.15.93:0/397509388 shutdown_connections 2023-12-14T18:30:19.620 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 1 -- 172.21.15.93:0/397509388 wait complete. 2023-12-14T18:30:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.618+0000 7f5715d24b80 20 client.25448 trim_cache size 0 max 16384 2023-12-14T18:30:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:30:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:30:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:30:19.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:30:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 -- 172.21.15.93:0/2584586047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56f00a12c0 msgr2=0x7f56f00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56f00a12c0 0x7f56f00a3770 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f56f400f540 tx=0x7f56f400b040 comp rx=0 tx=0).stop 2023-12-14T18:30:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 -- 172.21.15.93:0/2584586047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57040ffd60 msgr2=0x7f570410ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57040ffd60 0x7f570410ce20 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f56ec009fd0 tx=0x7f56ec00f690 comp rx=0 tx=0).stop 2023-12-14T18:30:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 -- 172.21.15.93:0/2584586047 shutdown_connections 2023-12-14T18:30:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56f00a12c0 0x7f56f00a3770 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5704104400 0x7f570410d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f57040ff3c0 0x7f570410c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 --2- 172.21.15.93:0/2584586047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f57040ffd60 0x7f570410ce20 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 -- 172.21.15.93:0/2584586047 >> 172.21.15.93:0/2584586047 conn(0x7f57040763a0 msgr2=0x7f57040fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 -- 172.21.15.93:0/2584586047 shutdown_connections 2023-12-14T18:30:19.623 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:19.619+0000 7f5709b96700 1 -- 172.21.15.93:0/2584586047 wait complete. 2023-12-14T18:30:20.000 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:19 smithi160 ceph-mon[43840]: pgmap v118: 129 pgs: 129 active+clean; 332 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.3 MiB/s wr, 44 op/s 2023-12-14T18:30:20.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:19 smithi093 ceph-mon[42070]: pgmap v118: 129 pgs: 129 active+clean; 332 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.3 MiB/s wr, 44 op/s 2023-12-14T18:30:20.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:19 smithi097 ceph-mon[43486]: pgmap v118: 129 pgs: 129 active+clean; 332 MiB data, 727 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.3 MiB/s wr, 44 op/s 2023-12-14T18:30:21.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:21 smithi160 ceph-mon[43840]: pgmap v119: 129 pgs: 129 active+clean; 360 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 255 op/s 2023-12-14T18:30:22.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:21 smithi093 ceph-mon[42070]: pgmap v119: 129 pgs: 129 active+clean; 360 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 255 op/s 2023-12-14T18:30:22.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:21 smithi097 ceph-mon[43486]: pgmap v119: 129 pgs: 129 active+clean; 360 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 14 MiB/s rd, 14 MiB/s wr, 255 op/s 2023-12-14T18:30:23.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:23 smithi160 ceph-mon[43840]: pgmap v120: 129 pgs: 129 active+clean; 360 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.7 MiB/s wr, 230 op/s 2023-12-14T18:30:24.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:23 smithi093 ceph-mon[42070]: pgmap v120: 129 pgs: 129 active+clean; 360 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.7 MiB/s wr, 230 op/s 2023-12-14T18:30:24.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:23 smithi097 ceph-mon[43486]: pgmap v120: 129 pgs: 129 active+clean; 360 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.7 MiB/s wr, 230 op/s 2023-12-14T18:30:25.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:25 smithi160 ceph-mon[43840]: pgmap v121: 129 pgs: 129 active+clean; 360 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.7 MiB/s wr, 230 op/s 2023-12-14T18:30:26.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:25 smithi093 ceph-mon[42070]: pgmap v121: 129 pgs: 129 active+clean; 360 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.7 MiB/s wr, 230 op/s 2023-12-14T18:30:26.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:25 smithi097 ceph-mon[43486]: pgmap v121: 129 pgs: 129 active+clean; 360 MiB data, 785 MiB used, 1.0 TiB / 1.0 TiB avail; 9.5 MiB/s rd, 9.7 MiB/s wr, 230 op/s 2023-12-14T18:30:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:27 smithi160 ceph-mon[43840]: pgmap v122: 129 pgs: 129 active+clean; 393 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 639 op/s 2023-12-14T18:30:28.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:27 smithi093 ceph-mon[42070]: pgmap v122: 129 pgs: 129 active+clean; 393 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 639 op/s 2023-12-14T18:30:28.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:27 smithi097 ceph-mon[43486]: pgmap v122: 129 pgs: 129 active+clean; 393 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 13 MiB/s wr, 639 op/s 2023-12-14T18:30:29.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:29 smithi160 ceph-mon[43840]: pgmap v123: 129 pgs: 129 active+clean; 393 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 620 op/s 2023-12-14T18:30:30.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:29 smithi093 ceph-mon[42070]: pgmap v123: 129 pgs: 129 active+clean; 393 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 620 op/s 2023-12-14T18:30:30.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:29 smithi097 ceph-mon[43486]: pgmap v123: 129 pgs: 129 active+clean; 393 MiB data, 896 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.8 MiB/s wr, 620 op/s 2023-12-14T18:30:31.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:31 smithi160 ceph-mon[43840]: pgmap v124: 129 pgs: 129 active+clean; 433 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 637 op/s 2023-12-14T18:30:32.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:31 smithi093 ceph-mon[42070]: pgmap v124: 129 pgs: 129 active+clean; 433 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 637 op/s 2023-12-14T18:30:32.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:31 smithi097 ceph-mon[43486]: pgmap v124: 129 pgs: 129 active+clean; 433 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 11 MiB/s wr, 637 op/s 2023-12-14T18:30:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.782+0000 7faaed7ca700 1 -- 172.21.15.93:0/473947420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae80ffb40 msgr2=0x7faae8104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.782+0000 7faaed7ca700 1 --2- 172.21.15.93:0/473947420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae80ffb40 0x7faae8104010 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7faad80099c0 tx=0x7faad8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.782+0000 7faaed7ca700 1 -- 172.21.15.93:0/473947420 shutdown_connections 2023-12-14T18:30:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.782+0000 7faaed7ca700 1 --2- 172.21.15.93:0/473947420 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae8104630 0x7faae8104ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.782+0000 7faaed7ca700 1 --2- 172.21.15.93:0/473947420 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae80ff230 0x7faae80ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.782+0000 7faaed7ca700 1 --2- 172.21.15.93:0/473947420 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae80ffb40 0x7faae8104010 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.782+0000 7faaed7ca700 1 -- 172.21.15.93:0/473947420 >> 172.21.15.93:0/473947420 conn(0x7faae80763a0 msgr2=0x7faae80767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.786+0000 7faaed7ca700 1 -- 172.21.15.93:0/473947420 shutdown_connections 2023-12-14T18:30:32.787 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.786+0000 7faaed7ca700 1 -- 172.21.15.93:0/473947420 wait complete. 2023-12-14T18:30:32.789 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.787+0000 7faaed7ca700 1 Processor -- start 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.787+0000 7faaed7ca700 1 -- start start 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.787+0000 7faaed7ca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae80ff230 0x7faae810c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.787+0000 7faaed7ca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae80ffb40 0x7faae810c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faaed7ca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae8104630 0x7faae81b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faaed7ca700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae806a510 con 0x7faae80ff230 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faaed7ca700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae806a680 con 0x7faae80ffb40 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faaed7ca700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faae806a840 con 0x7faae8104630 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae8104630 0x7faae81b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae77fe700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae8104630 0x7faae81b0f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:55296/0 (socket says 172.21.15.93:55296) 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae77fe700 1 -- 172.21.15.93:0/1033785938 learned_addr learned my addr 172.21.15.93:0/1033785938 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae67fc700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae80ffb40 0x7faae810c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae6ffd700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae80ff230 0x7faae810c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae67fc700 1 -- 172.21.15.93:0/1033785938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae8104630 msgr2=0x7faae81b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:32.790 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae67fc700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae8104630 0x7faae81b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae67fc700 1 -- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae80ff230 msgr2=0x7faae810c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae67fc700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae80ff230 0x7faae810c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.788+0000 7faae67fc700 1 -- 172.21.15.93:0/1033785938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faad80095f0 con 0x7faae80ffb40 2023-12-14T18:30:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.789+0000 7faae6ffd700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae80ff230 0x7faae810c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:30:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.789+0000 7faae67fc700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae80ffb40 0x7faae810c650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faad80098e0 tx=0x7faad800f690 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.790+0000 7faacffff700 1 -- 172.21.15.93:0/1033785938 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad801d070 con 0x7faae80ffb40 2023-12-14T18:30:32.791 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.790+0000 7faacffff700 1 -- 172.21.15.93:0/1033785938 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faad8005510 con 0x7faae80ffb40 2023-12-14T18:30:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.790+0000 7faaed7ca700 1 -- 172.21.15.93:0/1033785938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faae8116940 con 0x7faae80ffb40 2023-12-14T18:30:32.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.790+0000 7faacffff700 1 -- 172.21.15.93:0/1033785938 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8004b30 con 0x7faae80ffb40 2023-12-14T18:30:32.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.790+0000 7faaed7ca700 1 -- 172.21.15.93:0/1033785938 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae81167b0 con 0x7faae80ffb40 2023-12-14T18:30:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.795+0000 7faaf9958b80 1 Processor -- start 2023-12-14T18:30:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.795+0000 7faaf9958b80 1 -- start start 2023-12-14T18:30:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.795+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432939430 0x556432adbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.795+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432adc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.795+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432ae70f0 0x556432ae94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.795+0000 7faaf9958b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564329c6580 con 0x556432ae70f0 2023-12-14T18:30:32.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.795+0000 7faaf9958b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556432a35810 con 0x556432939430 2023-12-14T18:30:32.797 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.795+0000 7faaf9958b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556432a2eca0 con 0x556432adc370 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.795+0000 7faae6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432adc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae6ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432adc7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:55302/0 (socket says 172.21.15.93:55302) 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae6ffd700 1 -- 172.21.15.93:0/486920605 learned_addr learned my addr 172.21.15.93:0/486920605 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae77fe700 1 --2- 172.21.15.93:0/486920605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432939430 0x556432adbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faacffff700 1 -- 172.21.15.93:0/1033785938 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7faad800fb80 con 0x7faae80ffb40 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae67fc700 1 --2- 172.21.15.93:0/486920605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432ae70f0 0x556432ae94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae77fe700 1 -- 172.21.15.93:0/486920605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 msgr2=0x556432adc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae77fe700 1 --2- 172.21.15.93:0/486920605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432adc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae77fe700 1 -- 172.21.15.93:0/486920605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432ae70f0 msgr2=0x556432ae94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:32.798 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae77fe700 1 --2- 172.21.15.93:0/486920605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432ae70f0 0x556432ae94d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae77fe700 1 -- 172.21.15.93:0/486920605 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5564328b0350 con 0x556432939430 2023-12-14T18:30:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.796+0000 7faae77fe700 1 --2- 172.21.15.93:0/486920605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432939430 0x556432adbe30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faae0009fb0 tx=0x7faae0009710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a632ce200a34e1b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.797+0000 7faacd7fa700 1 -- 172.21.15.93:0/486920605 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae000ee40 con 0x556432939430 2023-12-14T18:30:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.797+0000 7faacd7fa700 1 -- 172.21.15.93:0/486920605 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faae000f040 con 0x556432939430 2023-12-14T18:30:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.797+0000 7faacd7fa700 1 -- 172.21.15.93:0/486920605 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faae0013740 con 0x556432939430 2023-12-14T18:30:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.797+0000 7faacffff700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7faad00a1490 0x7faad00a3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.797+0000 7faacffff700 1 -- 172.21.15.93:0/1033785938 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7faad80cdcb0 con 0x7faae80ffb40 2023-12-14T18:30:32.799 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.798+0000 7faaf9958b80 1 -- 172.21.15.93:0/486920605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432939430 msgr2=0x556432adbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.798+0000 7faaf9958b80 1 --2- 172.21.15.93:0/486920605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432939430 0x556432adbe30 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7faae0009fb0 tx=0x7faae0009710 comp rx=0 tx=0).stop 2023-12-14T18:30:32.800 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.798+0000 7faae6ffd700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7faad00a1490 0x7faad00a3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.799+0000 7faaf9958b80 1 -- 172.21.15.93:0/486920605 shutdown_connections 2023-12-14T18:30:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.799+0000 7faaf9958b80 1 --2- 172.21.15.93:0/486920605 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432ae70f0 0x556432ae94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.799+0000 7faaf9958b80 1 --2- 172.21.15.93:0/486920605 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432939430 0x556432adbe30 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.801 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.799+0000 7faaf9958b80 1 --2- 172.21.15.93:0/486920605 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432adc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.799+0000 7faaf9958b80 1 -- 172.21.15.93:0/486920605 >> 172.21.15.93:0/486920605 conn(0x556432ad2890 msgr2=0x556432ad8200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.801+0000 7faaf9958b80 1 -- 172.21.15.93:0/486920605 shutdown_connections 2023-12-14T18:30:32.802 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.802+0000 7faaf9958b80 1 -- 172.21.15.93:0/486920605 wait complete. 2023-12-14T18:30:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.802+0000 7faaf9958b80 1 -- start start 2023-12-14T18:30:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.802+0000 7faaf9958b80 1 Processor -- start 2023-12-14T18:30:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.802+0000 7faae6ffd700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7faad00a1490 0x7faad00a3940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faadc009fc0 tx=0x7faadc009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:32.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.803+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432ae69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.803+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432ae3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432ae70f0 0x556432ae41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faaf9958b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564329caf30 con 0x556432939430 2023-12-14T18:30:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faaf9958b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564329cabd0 con 0x556432ae70f0 2023-12-14T18:30:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faaf9958b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x5564328b2640 con 0x556432adc370 2023-12-14T18:30:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faaf9958b80 10 client.0 _ll_register_callbacks cb 0x5564329ee8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faae67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432ae3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faae67fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432ae3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:55304/0 (socket says 172.21.15.93:55304) 2023-12-14T18:30:32.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faae67fc700 1 -- 172.21.15.93:0/4013260802 learned_addr learned my addr 172.21.15.93:0/4013260802 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faae67fc700 1 -- 172.21.15.93:0/4013260802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432ae70f0 msgr2=0x556432ae41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faae67fc700 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432ae70f0 0x556432ae41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faae67fc700 1 -- 172.21.15.93:0/4013260802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 msgr2=0x556432ae69f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:30:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faae67fc700 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432ae69f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faae67fc700 1 -- 172.21.15.93:0/4013260802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faae0009c60 con 0x556432adc370 2023-12-14T18:30:32.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.804+0000 7faae67fc700 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432ae3ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faad8000c00 tx=0x7faad809a9a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=32dc12c9513b4237 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.805+0000 7faacaff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2023-12-14T18:30:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.805+0000 7faacaff5700 1 -- 172.21.15.93:0/4013260802 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad809a1b0 con 0x556432adc370 2023-12-14T18:30:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.805+0000 7faacaff5700 1 -- 172.21.15.93:0/4013260802 <== mon.1 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7faad809aac0 con 0x556432adc370 2023-12-14T18:30:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.805+0000 7faaf9958b80 1 -- 172.21.15.93:0/4013260802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x556432acfea0 con 0x556432adc370 2023-12-14T18:30:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.805+0000 7faacaff5700 1 -- 172.21.15.93:0/4013260802 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad809eaf0 con 0x556432adc370 2023-12-14T18:30:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.805+0000 7faaf9958b80 10 client.25456 fetch_fsmap learned FSMap version 28 2023-12-14T18:30:32.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.806+0000 7faaf9958b80 1 -- 172.21.15.93:0/4013260802 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x5564329f5110 con 0x556432adc370 2023-12-14T18:30:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.806+0000 7faacaff5700 1 -- 172.21.15.93:0/4013260802 <== mon.1 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x556432acfea0 con 0x556432adc370 2023-12-14T18:30:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.807+0000 7faacaff5700 1 -- 172.21.15.93:0/4013260802 <== mon.1 v2:172.21.15.160:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7faad8099a20 con 0x556432adc370 2023-12-14T18:30:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.807+0000 7faaf9958b80 10 client.25456 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:30:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.807+0000 7faaf9958b80 10 client.25456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:30:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.807+0000 7faaf9958b80 20 client.25456 populate_metadata read hostname 'smithi093' 2023-12-14T18:30:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.807+0000 7faaf9958b80 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x556432af67f0 0x556432af8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.807+0000 7faaf9958b80 4 client.25456 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.807+0000 7faaf9958b80 1 -- 172.21.15.93:0/4013260802 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x556432a27c80 con 0x556432af67f0 2023-12-14T18:30:32.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.807+0000 7faae6ffd700 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x556432af67f0 0x556432af8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.810+0000 7faae6ffd700 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x556432af67f0 0x556432af8c00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=990979e796211bf0 server_cookie=edf19d64eb65f467 in_seq=0 out_seq=0 2023-12-14T18:30:32.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.810+0000 7faacaff5700 10 client.25456 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.814+0000 7faacaff5700 1 -- 172.21.15.93:0/4013260802 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x556432a27c80 con 0x556432af67f0 2023-12-14T18:30:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.814+0000 7faacaff5700 10 client.25456 handle_command_reply: tid=0 2023-12-14T18:30:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 client.25456 shutdown 2023-12-14T18:30:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 -- 172.21.15.93:0/4013260802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 msgr2=0x556432ae3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432ae3ca0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7faad8000c00 tx=0x7faad809a9a0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 -- 172.21.15.93:0/4013260802 shutdown_connections 2023-12-14T18:30:32.816 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x556432af67f0 0x556432af8c00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432ae69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432ae70f0 0x556432ae41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 --2- 172.21.15.93:0/4013260802 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432adc370 0x556432ae3ca0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 -- 172.21.15.93:0/4013260802 >> 172.21.15.93:0/4013260802 conn(0x556432ad2890 msgr2=0x556432ad7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faacaff5700 0 client.25456 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 -- 172.21.15.93:0/4013260802 shutdown_connections 2023-12-14T18:30:32.817 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.815+0000 7faaf9958b80 1 -- 172.21.15.93:0/4013260802 wait complete. 2023-12-14T18:30:32.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.821+0000 7faaf9958b80 20 client.25456 trim_cache size 0 max 16384 2023-12-14T18:30:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.850+0000 7faaf9958b80 1 Processor -- start 2023-12-14T18:30:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.850+0000 7faaf9958b80 1 -- start start 2023-12-14T18:30:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.850+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 0x556432ae2b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.850+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432ae64b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.850+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432adc370 0x556432ae69f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.850+0000 7faaf9958b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556432ad7060 con 0x556432939430 2023-12-14T18:30:32.851 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.850+0000 7faaf9958b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556432ae94d0 con 0x556432adc370 2023-12-14T18:30:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.850+0000 7faaf9958b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556432add0d0 con 0x556432ad2890 2023-12-14T18:30:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.851+0000 7faae67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432ae64b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.851+0000 7faae67fc700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432ae64b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:52298/0 (socket says 172.21.15.93:52298) 2023-12-14T18:30:32.852 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.851+0000 7faae67fc700 1 -- 172.21.15.93:0/554528156 learned_addr learned my addr 172.21.15.93:0/554528156 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.851+0000 7faae67fc700 1 -- 172.21.15.93:0/554528156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 msgr2=0x556432ae2b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:30:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.851+0000 7faae67fc700 1 --2- 172.21.15.93:0/554528156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 0x556432ae2b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.851+0000 7faae67fc700 1 -- 172.21.15.93:0/554528156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432adc370 msgr2=0x556432ae69f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-14T18:30:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.851+0000 7faae67fc700 1 --2- 172.21.15.93:0/554528156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432adc370 0x556432ae69f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.851+0000 7faae67fc700 1 -- 172.21.15.93:0/554528156 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x556432ad40d0 con 0x556432939430 2023-12-14T18:30:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.852+0000 7faae67fc700 1 --2- 172.21.15.93:0/554528156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432ae64b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7faad80a2950 tx=0x7faad8009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d0fc8b1fa675fc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.852+0000 7faacdffb700 1 -- 172.21.15.93:0/554528156 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad809aac0 con 0x556432939430 2023-12-14T18:30:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.852+0000 7faacdffb700 1 -- 172.21.15.93:0/554528156 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faad809eaf0 con 0x556432939430 2023-12-14T18:30:32.853 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.852+0000 7faacdffb700 1 -- 172.21.15.93:0/554528156 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad80ba2a0 con 0x556432939430 2023-12-14T18:30:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.853+0000 7faaf9958b80 1 -- 172.21.15.93:0/554528156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 msgr2=0x556432ae64b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:32.854 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.853+0000 7faaf9958b80 1 --2- 172.21.15.93:0/554528156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432ae64b0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7faad80a2950 tx=0x7faad8009310 comp rx=0 tx=0).stop 2023-12-14T18:30:32.856 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.856+0000 7faaf9958b80 1 -- 172.21.15.93:0/554528156 shutdown_connections 2023-12-14T18:30:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.856+0000 7faaf9958b80 1 --2- 172.21.15.93:0/554528156 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432adc370 0x556432ae69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.856+0000 7faaf9958b80 1 --2- 172.21.15.93:0/554528156 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 0x556432ae2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.856+0000 7faaf9958b80 1 --2- 172.21.15.93:0/554528156 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432ae64b0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.857 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.856+0000 7faaf9958b80 1 -- 172.21.15.93:0/554528156 >> 172.21.15.93:0/554528156 conn(0x556432ad6c90 msgr2=0x556432ae2450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.857+0000 7faaf9958b80 1 -- 172.21.15.93:0/554528156 shutdown_connections 2023-12-14T18:30:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.857+0000 7faaf9958b80 1 -- 172.21.15.93:0/554528156 wait complete. 2023-12-14T18:30:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.857+0000 7faaf9958b80 1 -- start start 2023-12-14T18:30:32.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.857+0000 7faaf9958b80 1 Processor -- start 2023-12-14T18:30:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432aeb770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 0x556432aebcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faaf9958b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432adc370 0x556432b496b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faaf9958b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x556432aec570 con 0x556432939430 2023-12-14T18:30:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faaf9958b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x556432b49bf0 con 0x556432adc370 2023-12-14T18:30:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faaf9958b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x556432b49d30 con 0x556432ad2890 2023-12-14T18:30:32.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faaf9958b80 10 client.0 _ll_register_callbacks cb 0x5564329ee8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faae6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432aeb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faae6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432aeb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:52302/0 (socket says 172.21.15.93:52302) 2023-12-14T18:30:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faae6ffd700 1 -- 172.21.15.93:0/483167349 learned_addr learned my addr 172.21.15.93:0/483167349 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faae67fc700 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 0x556432aebcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.858+0000 7faae77fe700 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432adc370 0x556432b496b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.859+0000 7faae67fc700 1 -- 172.21.15.93:0/483167349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432adc370 msgr2=0x556432b496b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.859+0000 7faae67fc700 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432adc370 0x556432b496b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.859+0000 7faae67fc700 1 -- 172.21.15.93:0/483167349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 msgr2=0x556432aeb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.859+0000 7faae67fc700 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432aeb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.859+0000 7faae67fc700 1 -- 172.21.15.93:0/483167349 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5564329f5110 con 0x556432ad2890 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.859+0000 7faae67fc700 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 0x556432aebcb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faad80a2950 tx=0x7faad809ec60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60fa8a153d1654ca server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.859+0000 7faacaff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.859+0000 7faacaff5700 1 -- 172.21.15.93:0/483167349 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faad8099fb0 con 0x556432ad2890 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.860+0000 7faaf9958b80 1 -- 172.21.15.93:0/483167349 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7faae000f040 con 0x556432ad2890 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.860+0000 7faacaff5700 1 -- 172.21.15.93:0/483167349 <== mon.1 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7faad809a110 con 0x556432ad2890 2023-12-14T18:30:32.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.860+0000 7faacaff5700 1 -- 172.21.15.93:0/483167349 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) v2 ==== 24+0+0 (secure 0 0 0) 0x7faad80968b0 con 0x556432ad2890 2023-12-14T18:30:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faaf9958b80 10 client.25459 fetch_fsmap learned FSMap version 28 2023-12-14T18:30:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faaf9958b80 1 -- 172.21.15.93:0/483167349 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) v3 -- 0x55643285cb00 con 0x556432ad2890 2023-12-14T18:30:32.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faacaff5700 1 -- 172.21.15.93:0/483167349 <== mon.1 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7faae000f040 con 0x556432ad2890 2023-12-14T18:30:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faacaff5700 1 -- 172.21.15.93:0/483167349 <== mon.1 v2:172.21.15.160:3300/0 5 ==== fsmap(e 28) v1 ==== 7486+0+0 (secure 0 0 0) 0x7faad809b330 con 0x556432ad2890 2023-12-14T18:30:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faaf9958b80 10 client.25459 fetch_fsmap finished waiting for FSMap version 28 2023-12-14T18:30:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faaf9958b80 10 client.25459 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:30:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faaf9958b80 20 client.25459 populate_metadata read hostname 'smithi093' 2023-12-14T18:30:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faaf9958b80 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x556432ae70f0 0x556432b4cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faaf9958b80 4 client.25459 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:30:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.861+0000 7faaf9958b80 1 -- 172.21.15.93:0/483167349 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x556432a279a0 con 0x556432ae70f0 2023-12-14T18:30:32.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.862+0000 7faae6ffd700 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x556432ae70f0 0x556432b4cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.871+0000 7faae6ffd700 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x556432ae70f0 0x556432b4cf10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=523da245cfdc1531 server_cookie=b27e2411a4ee25d6 in_seq=0 out_seq=0 2023-12-14T18:30:32.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:32.871+0000 7faacaff5700 10 client.25459 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:33.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:33 smithi160 ceph-mon[43840]: pgmap v125: 129 pgs: 129 active+clean; 433 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 425 op/s 2023-12-14T18:30:34.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:33 smithi093 ceph-mon[42070]: pgmap v125: 129 pgs: 129 active+clean; 433 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 425 op/s 2023-12-14T18:30:34.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:33 smithi097 ceph-mon[43486]: pgmap v125: 129 pgs: 129 active+clean; 433 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 425 op/s 2023-12-14T18:30:35.998 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:35 smithi160 ceph-mon[43840]: pgmap v126: 129 pgs: 129 active+clean; 433 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 425 op/s 2023-12-14T18:30:36.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:35 smithi093 ceph-mon[42070]: pgmap v126: 129 pgs: 129 active+clean; 433 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 425 op/s 2023-12-14T18:30:36.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:35 smithi097 ceph-mon[43486]: pgmap v126: 129 pgs: 129 active+clean; 433 MiB data, 900 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 MiB/s rd, 6.1 MiB/s wr, 425 op/s 2023-12-14T18:30:37.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:37 smithi160 ceph-mon[43840]: pgmap v127: 129 pgs: 129 active+clean; 479 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 482 op/s 2023-12-14T18:30:38.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:37 smithi093 ceph-mon[42070]: pgmap v127: 129 pgs: 129 active+clean; 479 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 482 op/s 2023-12-14T18:30:38.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:37 smithi097 ceph-mon[43486]: pgmap v127: 129 pgs: 129 active+clean; 479 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 10 MiB/s wr, 482 op/s 2023-12-14T18:30:39.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:39 smithi160 ceph-mon[43840]: pgmap v128: 129 pgs: 129 active+clean; 479 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 73 op/s 2023-12-14T18:30:40.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:39 smithi093 ceph-mon[42070]: pgmap v128: 129 pgs: 129 active+clean; 479 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 73 op/s 2023-12-14T18:30:40.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:39 smithi097 ceph-mon[43486]: pgmap v128: 129 pgs: 129 active+clean; 479 MiB data, 914 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 7.2 MiB/s wr, 73 op/s 2023-12-14T18:30:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.102+0000 7faacaff5700 1 -- 172.21.15.93:0/483167349 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x556432a279a0 con 0x556432ae70f0 2023-12-14T18:30:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.102+0000 7faacaff5700 10 client.25459 handle_command_reply: tid=0 2023-12-14T18:30:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.102+0000 7faaf9958b80 1 client.25459 shutdown 2023-12-14T18:30:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.102+0000 7faaf9958b80 1 -- 172.21.15.93:0/483167349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 msgr2=0x556432aebcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:41.106 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faaf9958b80 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 0x556432aebcb0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faad80a2950 tx=0x7faad809ec60 comp rx=0 tx=0).stop 2023-12-14T18:30:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faaf9958b80 1 -- 172.21.15.93:0/483167349 shutdown_connections 2023-12-14T18:30:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faaf9958b80 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x556432ae70f0 0x556432b4cf10 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faaf9958b80 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x556432939430 0x556432aeb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faaf9958b80 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x556432adc370 0x556432b496b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faaf9958b80 1 --2- 172.21.15.93:0/483167349 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x556432ad2890 0x556432aebcb0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faaf9958b80 1 -- 172.21.15.93:0/483167349 >> 172.21.15.93:0/483167349 conn(0x556432ad6c90 msgr2=0x556432ae2ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faacaff5700 0 client.25459 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:41.107 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faaf9958b80 1 -- 172.21.15.93:0/483167349 shutdown_connections 2023-12-14T18:30:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.103+0000 7faaf9958b80 1 -- 172.21.15.93:0/483167349 wait complete. 2023-12-14T18:30:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.104+0000 7faaf9958b80 20 client.25459 trim_cache size 0 max 16384 2023-12-14T18:30:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:30:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:30:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:30:41.108 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:30:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.104+0000 7faaed7ca700 1 -- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7faad00a1490 msgr2=0x7faad00a3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:41.109 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.104+0000 7faaed7ca700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7faad00a1490 0x7faad00a3940 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7faadc009fc0 tx=0x7faadc009260 comp rx=0 tx=0).stop 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.104+0000 7faaed7ca700 1 -- 172.21.15.93:0/1033785938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae80ffb40 msgr2=0x7faae810c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.104+0000 7faaed7ca700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae80ffb40 0x7faae810c650 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7faad80098e0 tx=0x7faad800f690 comp rx=0 tx=0).stop 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.104+0000 7faaed7ca700 1 -- 172.21.15.93:0/1033785938 shutdown_connections 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.104+0000 7faaed7ca700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7faad00a1490 0x7faad00a3940 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.104+0000 7faaed7ca700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7faae80ff230 0x7faae810c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.105+0000 7faaed7ca700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7faae8104630 0x7faae81b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.105+0000 7faaed7ca700 1 --2- 172.21.15.93:0/1033785938 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7faae80ffb40 0x7faae810c650 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.105+0000 7faaed7ca700 1 -- 172.21.15.93:0/1033785938 >> 172.21.15.93:0/1033785938 conn(0x7faae80763a0 msgr2=0x7faae8102050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.105+0000 7faaed7ca700 1 -- 172.21.15.93:0/1033785938 shutdown_connections 2023-12-14T18:30:41.110 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:41.105+0000 7faaed7ca700 1 -- 172.21.15.93:0/1033785938 wait complete. 2023-12-14T18:30:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:41 smithi160 ceph-mon[43840]: pgmap v129: 129 pgs: 129 active+clean; 464 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 1.63k op/s 2023-12-14T18:30:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:41 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:30:42.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:41 smithi093 ceph-mon[42070]: pgmap v129: 129 pgs: 129 active+clean; 464 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 1.63k op/s 2023-12-14T18:30:42.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:41 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:30:42.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:41 smithi097 ceph-mon[43486]: pgmap v129: 129 pgs: 129 active+clean; 464 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 9.4 MiB/s wr, 1.63k op/s 2023-12-14T18:30:42.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:41 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:30:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:43 smithi160 ceph-mon[43840]: pgmap v130: 129 pgs: 129 active+clean; 464 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.2 MiB/s wr, 1.61k op/s 2023-12-14T18:30:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:43 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:43 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:43 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:43 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:43 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:30:43.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:43 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:30:43.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:43 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:43 smithi093 ceph-mon[42070]: pgmap v130: 129 pgs: 129 active+clean; 464 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.2 MiB/s wr, 1.61k op/s 2023-12-14T18:30:43.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:43 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:43 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:43 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:43 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:43 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:30:43.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:43 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:30:43.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:43 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:43 smithi097 ceph-mon[43486]: pgmap v130: 129 pgs: 129 active+clean; 464 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.2 MiB/s wr, 1.61k op/s 2023-12-14T18:30:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:43 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:43 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:43 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:43 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:43 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:30:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:43 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:30:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:43 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:30:44.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:30:44.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:44 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:30:44.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:44 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:30:44.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:30:44.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:30:44.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:44 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:30:44.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:44 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:30:44.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:30:44.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:30:44.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:44 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:30:44.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:44 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:30:44.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:30:45.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:45 smithi160 ceph-mon[43840]: pgmap v131: 129 pgs: 129 active+clean; 464 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.2 MiB/s wr, 1.61k op/s 2023-12-14T18:30:45.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:45 smithi093 ceph-mon[42070]: pgmap v131: 129 pgs: 129 active+clean; 464 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.2 MiB/s wr, 1.61k op/s 2023-12-14T18:30:45.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:45 smithi097 ceph-mon[43486]: pgmap v131: 129 pgs: 129 active+clean; 464 MiB data, 998 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 6.2 MiB/s wr, 1.61k op/s 2023-12-14T18:30:47.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:47 smithi160 ceph-mon[43840]: pgmap v132: 129 pgs: 129 active+clean; 450 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 1.72k op/s 2023-12-14T18:30:47.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:47 smithi093 ceph-mon[42070]: pgmap v132: 129 pgs: 129 active+clean; 450 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 1.72k op/s 2023-12-14T18:30:47.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:47 smithi097 ceph-mon[43486]: pgmap v132: 129 pgs: 129 active+clean; 450 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 8.4 MiB/s wr, 1.72k op/s 2023-12-14T18:30:49.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:49 smithi160 ceph-mon[43840]: pgmap v133: 129 pgs: 129 active+clean; 450 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 1.66k op/s 2023-12-14T18:30:49.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:49 smithi093 ceph-mon[42070]: pgmap v133: 129 pgs: 129 active+clean; 450 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 1.66k op/s 2023-12-14T18:30:49.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:49 smithi097 ceph-mon[43486]: pgmap v133: 129 pgs: 129 active+clean; 450 MiB data, 997 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 MiB/s rd, 4.5 MiB/s wr, 1.66k op/s 2023-12-14T18:30:51.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:51 smithi160 ceph-mon[43840]: pgmap v134: 129 pgs: 129 active+clean; 485 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.5 MiB/s wr, 1.79k op/s 2023-12-14T18:30:51.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:51 smithi093 ceph-mon[42070]: pgmap v134: 129 pgs: 129 active+clean; 485 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.5 MiB/s wr, 1.79k op/s 2023-12-14T18:30:51.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:51 smithi097 ceph-mon[43486]: pgmap v134: 129 pgs: 129 active+clean; 485 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 MiB/s rd, 7.5 MiB/s wr, 1.79k op/s 2023-12-14T18:30:53.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:53 smithi160 ceph-mon[43840]: pgmap v135: 129 pgs: 129 active+clean; 485 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2023-12-14T18:30:53.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:53 smithi093 ceph-mon[42070]: pgmap v135: 129 pgs: 129 active+clean; 485 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2023-12-14T18:30:53.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:53 smithi097 ceph-mon[43486]: pgmap v135: 129 pgs: 129 active+clean; 485 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2023-12-14T18:30:54.229 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.225+0000 7ff4a3c38700 1 -- 172.21.15.93:0/2971398890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ff3c0 msgr2=0x7ff49c0ff790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.225+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/2971398890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ff3c0 0x7ff49c0ff790 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7ff4980099c0 tx=0x7ff498009cd0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.225+0000 7ff4a3c38700 1 -- 172.21.15.93:0/2971398890 shutdown_connections 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.225+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/2971398890 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ff3c0 0x7ff49c0ff790 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.225+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/2971398890 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff49c104400 0x7ff49c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.225+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/2971398890 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49c0ffd60 0x7ff49c103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.225+0000 7ff4a3c38700 1 -- 172.21.15.93:0/2971398890 >> 172.21.15.93:0/2971398890 conn(0x7ff49c0763a0 msgr2=0x7ff49c0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.225+0000 7ff4a3c38700 1 -- 172.21.15.93:0/2971398890 shutdown_connections 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.225+0000 7ff4a3c38700 1 -- 172.21.15.93:0/2971398890 wait complete. 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a3c38700 1 Processor -- start 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a3c38700 1 -- start start 2023-12-14T18:30:54.230 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a3c38700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff49c0ff3c0 0x7ff49c195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a3c38700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ffd60 0x7ff49c1957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a3c38700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49c104400 0x7ff49c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a3c38700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19a270 con 0x7ff49c0ffd60 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a3c38700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19a3e0 con 0x7ff49c104400 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a3c38700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff49c19a5a0 con 0x7ff49c0ff3c0 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a11d3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ffd60 0x7ff49c1957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a11d3700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ffd60 0x7ff49c1957d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47324/0 (socket says 172.21.15.93:47324) 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a11d3700 1 -- 172.21.15.93:0/246492816 learned_addr learned my addr 172.21.15.93:0/246492816 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff49c0ff3c0 0x7ff49c195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.226+0000 7ff4a21d5700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49c104400 0x7ff49c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff4a11d3700 1 -- 172.21.15.93:0/246492816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff49c0ff3c0 msgr2=0x7ff49c195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff49c0ff3c0 0x7ff49c195290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.231 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff4a11d3700 1 -- 172.21.15.93:0/246492816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49c104400 msgr2=0x7ff49c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49c104400 0x7ff49c199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff4a11d3700 1 -- 172.21.15.93:0/246492816 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4980095f0 con 0x7ff49c0ffd60 2023-12-14T18:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff4a21d5700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49c104400 0x7ff49c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ffd60 0x7ff49c1957d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e940 tx=0x7ff48c00ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff492ffd700 1 -- 172.21.15.93:0/246492816 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c009730 con 0x7ff49c0ffd60 2023-12-14T18:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff492ffd700 1 -- 172.21.15.93:0/246492816 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff48c010430 con 0x7ff49c0ffd60 2023-12-14T18:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff4a3c38700 1 -- 172.21.15.93:0/246492816 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff49c10f210 con 0x7ff49c0ffd60 2023-12-14T18:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff492ffd700 1 -- 172.21.15.93:0/246492816 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c00cea0 con 0x7ff49c0ffd60 2023-12-14T18:30:54.232 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.227+0000 7ff4a3c38700 1 -- 172.21.15.93:0/246492816 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c10f080 con 0x7ff49c0ffd60 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.229+0000 7ff492ffd700 1 -- 172.21.15.93:0/246492816 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7ff48c010810 con 0x7ff49c0ffd60 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.230+0000 7ff492ffd700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff4880a12c0 0x7ff4880a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.230+0000 7ff492ffd700 1 -- 172.21.15.93:0/246492816 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7ff48c014070 con 0x7ff49c0ffd60 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.230+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff4880a12c0 0x7ff4880a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.230+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff4880a12c0 0x7ff4880a3770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498005fb0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4afdc6b80 1 Processor -- start 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4afdc6b80 1 -- start start 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262e77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.238 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db261d1580 con 0x55db262f20f0 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db26240810 con 0x55db262e7370 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db26239ca0 con 0x55db26144430 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a19d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a21d5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a11d3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262e77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a19d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262e6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:33330/0 (socket says 172.21.15.93:33330) 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a19d4700 1 -- 172.21.15.93:0/1939330464 learned_addr learned my addr 172.21.15.93:0/1939330464 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a21d5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47326/0 (socket says 172.21.15.93:47326) 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a11d3700 1 -- 172.21.15.93:0/1939330464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 msgr2=0x55db262e6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/1939330464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262e6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a11d3700 1 -- 172.21.15.93:0/1939330464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 msgr2=0x55db262f44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.231+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/1939330464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.239 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.232+0000 7ff4a11d3700 1 -- 172.21.15.93:0/1939330464 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db260bb350 con 0x55db262e7370 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.232+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/1939330464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262e6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.232+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/1939330464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262e77e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0ca5d0 tx=0x7ff48c0998b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6aaee9da583c1e56 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.232+0000 7ff487fff700 1 -- 172.21.15.93:0/1939330464 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0a23a0 con 0x55db262e7370 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.232+0000 7ff487fff700 1 -- 172.21.15.93:0/1939330464 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff48c0982f0 con 0x55db262e7370 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.232+0000 7ff487fff700 1 -- 172.21.15.93:0/1939330464 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0a1f10 con 0x55db262e7370 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1939330464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 msgr2=0x55db262e77e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1939330464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262e77e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7ff48c0ca5d0 tx=0x7ff48c0998b0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1939330464 shutdown_connections 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1939330464 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1939330464 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262e6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1939330464 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262e77e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1939330464 >> 172.21.15.93:0/1939330464 conn(0x55db262dd890 msgr2=0x55db262e3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1939330464 shutdown_connections 2023-12-14T18:30:54.240 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1939330464 wait complete. 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 -- start start 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.233+0000 7ff4afdc6b80 1 Processor -- start 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262e7370 0x55db262f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262f20f0 0x55db262f1000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db261d5f30 con 0x55db26144430 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db261d5bd0 con 0x55db262f20f0 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db260bd640 con 0x55db262e7370 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4afdc6b80 10 client.0 _ll_register_callbacks cb 0x55db261f98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a19d4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a19d4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47330/0 (socket says 172.21.15.93:47330) 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a19d4700 1 -- 172.21.15.93:0/1678698162 learned_addr learned my addr 172.21.15.93:0/1678698162 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:54.241 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a21d5700 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262f20f0 0x55db262f1000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262e7370 0x55db262f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a11d3700 1 -- 172.21.15.93:0/1678698162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262f20f0 msgr2=0x55db262f1000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262f20f0 0x55db262f1000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a11d3700 1 -- 172.21.15.93:0/1678698162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 msgr2=0x55db262f9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f9890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a11d3700 1 -- 172.21.15.93:0/1678698162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49c10f080 con 0x55db262e7370 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262e7370 0x55db262f9dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff48c098540 tx=0x7ff48c010c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a56e05771ebf93d server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4857fa700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4857fa700 1 -- 172.21.15.93:0/1678698162 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff48c0b8070 con 0x55db262e7370 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1678698162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55db262daea0 con 0x55db262e7370 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.234+0000 7ff4857fa700 1 -- 172.21.15.93:0/1678698162 <== mon.1 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7ff48c0a23a0 con 0x55db262e7370 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4857fa700 1 -- 172.21.15.93:0/1678698162 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff48c0b81d0 con 0x55db262e7370 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4afdc6b80 10 client.25471 fetch_fsmap learned FSMap version 29 2023-12-14T18:30:54.242 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1678698162 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55db26200110 con 0x55db262e7370 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4857fa700 1 -- 172.21.15.93:0/1678698162 <== mon.1 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff48c063020 con 0x55db262e7370 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4857fa700 1 -- 172.21.15.93:0/1678698162 <== mon.1 v2:172.21.15.160:3300/0 5 ==== fsmap(e 29) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff48c0cad20 con 0x55db262e7370 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4afdc6b80 10 client.25471 fetch_fsmap finished waiting for FSMap version 29 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4afdc6b80 10 client.25471 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4afdc6b80 20 client.25471 populate_metadata read hostname 'smithi093' 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55db26303860 0x55db26305c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4afdc6b80 4 client.25471 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1678698162 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55db26232c80 con 0x55db26303860 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.235+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55db26303860 0x55db26305c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.236+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55db26303860 0x55db26305c40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c61ed9210e26bae server_cookie=cd0f88c0397b2a54 in_seq=0 out_seq=0 2023-12-14T18:30:54.243 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.236+0000 7ff4857fa700 10 client.25471 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4857fa700 1 -- 172.21.15.93:0/1678698162 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55db26232c80 con 0x55db26303860 2023-12-14T18:30:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4857fa700 10 client.25471 handle_command_reply: tid=0 2023-12-14T18:30:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 client.25471 shutdown 2023-12-14T18:30:54.244 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1678698162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262e7370 msgr2=0x55db262f9dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262e7370 0x55db262f9dd0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff48c098540 tx=0x7ff48c010c10 comp rx=0 tx=0).stop 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1678698162 shutdown_connections 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55db26303860 0x55db26305c40 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262f20f0 0x55db262f1000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/1678698162 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262e7370 0x55db262f9dd0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1678698162 >> 172.21.15.93:0/1678698162 conn(0x55db262dd890 msgr2=0x55db262e2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4857fa700 0 client.25471 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1678698162 shutdown_connections 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/1678698162 wait complete. 2023-12-14T18:30:54.245 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.240+0000 7ff4afdc6b80 20 client.25471 trim_cache size 0 max 16384 2023-12-14T18:30:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.255+0000 7ff4afdc6b80 1 Processor -- start 2023-12-14T18:30:54.258 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.255+0000 7ff4afdc6b80 1 -- start start 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262ec0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262f20f0 0x55db262ec5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db262daa70 con 0x55db26144430 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db262dd4d0 con 0x55db262e7370 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db26300670 con 0x55db262f20f0 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a19d4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a19d4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f1960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47334/0 (socket says 172.21.15.93:47334) 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a19d4700 1 -- 172.21.15.93:0/2169316823 learned_addr learned my addr 172.21.15.93:0/2169316823 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a21d5700 1 --2- 172.21.15.93:0/2169316823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262f20f0 0x55db262ec5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/2169316823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262ec0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a19d4700 1 -- 172.21.15.93:0/2169316823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262f20f0 msgr2=0x55db262ec5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.259 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/2169316823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262f20f0 0x55db262ec5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a19d4700 1 -- 172.21.15.93:0/2169316823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 msgr2=0x55db262ec0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/2169316823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262ec0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a19d4700 1 -- 172.21.15.93:0/2169316823 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55db261ffe60 con 0x55db26144430 2023-12-14T18:30:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/2169316823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f1960 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff498000c00 tx=0x7ff498022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c33d53595163fb7b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.256+0000 7ff490dac700 1 -- 172.21.15.93:0/2169316823 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff49802c040 con 0x55db26144430 2023-12-14T18:30:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff490dac700 1 -- 172.21.15.93:0/2169316823 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff49802d040 con 0x55db26144430 2023-12-14T18:30:54.260 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff490dac700 1 -- 172.21.15.93:0/2169316823 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff498004640 con 0x55db26144430 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/2169316823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 msgr2=0x55db262f1960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/2169316823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f1960 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff498000c00 tx=0x7ff498022000 comp rx=0 tx=0).stop 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/2169316823 shutdown_connections 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/2169316823 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db26144430 0x55db262f1960 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/2169316823 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db262f20f0 0x55db262ec5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/2169316823 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262ec0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/2169316823 >> 172.21.15.93:0/2169316823 conn(0x55db262dd890 msgr2=0x55db2630a560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/2169316823 shutdown_connections 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.257+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/2169316823 wait complete. 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4afdc6b80 1 -- start start 2023-12-14T18:30:54.263 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4afdc6b80 1 Processor -- start 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262f7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4afdc6b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f7e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db26358be0 con 0x55db262f20f0 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db26358d50 con 0x55db262e7370 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4afdc6b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55db26358f10 con 0x55db26144430 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4afdc6b80 10 client.0 _ll_register_callbacks cb 0x7ff494009440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a21d5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f7e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a11d3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262f7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a19d4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a21d5700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f7e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:47350/0 (socket says 172.21.15.93:47350) 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a21d5700 1 -- 172.21.15.93:0/3118981491 learned_addr learned my addr 172.21.15.93:0/3118981491 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a21d5700 1 -- 172.21.15.93:0/3118981491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 msgr2=0x55db262f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a21d5700 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262f73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.264 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a21d5700 1 -- 172.21.15.93:0/3118981491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 msgr2=0x55db262f7930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a21d5700 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262f7930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a21d5700 1 -- 172.21.15.93:0/3118981491 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff49800b810 con 0x55db262f20f0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.258+0000 7ff4a11d3700 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4a21d5700 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f7e70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff494004660 tx=0x7ff494004690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e052d29239b54c70 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4857fa700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4857fa700 1 -- 172.21.15.93:0/3118981491 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff494007a30 con 0x55db262f20f0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4857fa700 1 -- 172.21.15.93:0/3118981491 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7ff494007b90 con 0x55db262f20f0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/3118981491 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7ff48c0982f0 con 0x55db262f20f0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4857fa700 1 -- 172.21.15.93:0/3118981491 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff494007d50 con 0x55db262f20f0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4afdc6b80 10 client.15855 fetch_fsmap learned FSMap version 29 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/3118981491 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) v3 -- 0x55db261e6940 con 0x55db262f20f0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4857fa700 1 -- 172.21.15.93:0/3118981491 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7ff48c0982f0 con 0x55db262f20f0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4857fa700 1 -- 172.21.15.93:0/3118981491 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 29) v1 ==== 7490+0+0 (secure 0 0 0) 0x7ff494017530 con 0x55db262f20f0 2023-12-14T18:30:54.265 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4afdc6b80 10 client.15855 fetch_fsmap finished waiting for FSMap version 29 2023-12-14T18:30:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4afdc6b80 10 client.15855 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:30:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4afdc6b80 20 client.15855 populate_metadata read hostname 'smithi093' 2023-12-14T18:30:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55db2635a700 0x55db2635cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:30:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4afdc6b80 4 client.15855 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:30:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/3118981491 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55db262329a0 con 0x55db2635a700 2023-12-14T18:30:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.259+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55db2635a700 0x55db2635cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:30:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.260+0000 7ff4857fa700 10 client.15855 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:54.266 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:54.260+0000 7ff4a19d4700 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55db2635a700 0x55db2635cb10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67db0c3f1c9779fc server_cookie=41d432ad9d46b8c6 in_seq=0 out_seq=0 2023-12-14T18:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.328+0000 7ff4857fa700 1 -- 172.21.15.93:0/3118981491 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55db262329a0 con 0x55db2635a700 2023-12-14T18:30:55.330 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.328+0000 7ff4857fa700 10 client.15855 handle_command_reply: tid=0 2023-12-14T18:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.328+0000 7ff4afdc6b80 1 client.15855 shutdown 2023-12-14T18:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/3118981491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 msgr2=0x55db262f7e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f7e70 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7ff494004660 tx=0x7ff494004690 comp rx=0 tx=0).stop 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/3118981491 shutdown_connections 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55db2635a700 0x55db2635cb10 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55db262f20f0 0x55db262f7e70 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55db26144430 0x55db262f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 --2- 172.21.15.93:0/3118981491 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55db262e7370 0x55db262f7930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/3118981491 >> 172.21.15.93:0/3118981491 conn(0x55db262dd890 msgr2=0x55db262f0b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4857fa700 0 client.15855 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/3118981491 shutdown_connections 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 1 -- 172.21.15.93:0/3118981491 wait complete. 2023-12-14T18:30:55.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.329+0000 7ff4afdc6b80 20 client.15855 trim_cache size 0 max 16384 2023-12-14T18:30:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:30:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:30:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:30:55.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:30:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 -- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff4880a12c0 msgr2=0x7ff4880a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff4880a12c0 0x7ff4880a3770 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff498009990 tx=0x7ff498005fb0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 -- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ffd60 msgr2=0x7ff49c1957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:30:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ffd60 0x7ff49c1957d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff48c00e940 tx=0x7ff48c00ec50 comp rx=0 tx=0).stop 2023-12-14T18:30:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 -- 172.21.15.93:0/246492816 shutdown_connections 2023-12-14T18:30:55.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7ff49c104400 0x7ff49c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7ff49c0ff3c0 0x7ff49c195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7ff4880a12c0 0x7ff4880a3770 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 --2- 172.21.15.93:0/246492816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7ff49c0ffd60 0x7ff49c1957d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.331+0000 7ff4a3c38700 1 -- 172.21.15.93:0/246492816 >> 172.21.15.93:0/246492816 conn(0x7ff49c0763a0 msgr2=0x7ff49c0fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.332+0000 7ff4a3c38700 1 -- 172.21.15.93:0/246492816 shutdown_connections 2023-12-14T18:30:55.335 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:30:55.332+0000 7ff4a3c38700 1 -- 172.21.15.93:0/246492816 wait complete. 2023-12-14T18:30:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:55 smithi160 ceph-mon[43840]: pgmap v136: 129 pgs: 129 active+clean; 485 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2023-12-14T18:30:55.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:55 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:55.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:55 smithi093 ceph-mon[42070]: pgmap v136: 129 pgs: 129 active+clean; 485 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2023-12-14T18:30:55.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:55 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:55.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:55 smithi097 ceph-mon[43486]: pgmap v136: 129 pgs: 129 active+clean; 485 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 MiB/s rd, 5.2 MiB/s wr, 233 op/s 2023-12-14T18:30:55.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:55 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:30:57.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:57 smithi160 ceph-mon[43840]: pgmap v137: 129 pgs: 129 active+clean; 506 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 375 op/s 2023-12-14T18:30:57.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:57 smithi093 ceph-mon[42070]: pgmap v137: 129 pgs: 129 active+clean; 506 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 375 op/s 2023-12-14T18:30:57.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:57 smithi097 ceph-mon[43486]: pgmap v137: 129 pgs: 129 active+clean; 506 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 7.0 MiB/s rd, 6.9 MiB/s wr, 375 op/s 2023-12-14T18:30:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:30:59 smithi160 ceph-mon[43840]: pgmap v138: 129 pgs: 129 active+clean; 506 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2023-12-14T18:30:59.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:30:59 smithi093 ceph-mon[42070]: pgmap v138: 129 pgs: 129 active+clean; 506 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2023-12-14T18:30:59.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:30:59 smithi097 ceph-mon[43486]: pgmap v138: 129 pgs: 129 active+clean; 506 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 4.7 MiB/s wr, 266 op/s 2023-12-14T18:31:00.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:00 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:31:00.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:00 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:31:00.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:00 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:31:01.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:01 smithi160 ceph-mon[43840]: pgmap v139: 129 pgs: 129 active+clean; 474 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.1 MiB/s wr, 552 op/s 2023-12-14T18:31:01.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:01 smithi093 ceph-mon[42070]: pgmap v139: 129 pgs: 129 active+clean; 474 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.1 MiB/s wr, 552 op/s 2023-12-14T18:31:01.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:01 smithi097 ceph-mon[43486]: pgmap v139: 129 pgs: 129 active+clean; 474 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 MiB/s rd, 6.1 MiB/s wr, 552 op/s 2023-12-14T18:31:03.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:03 smithi160 ceph-mon[43840]: pgmap v140: 129 pgs: 129 active+clean; 474 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 428 op/s 2023-12-14T18:31:03.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:03 smithi093 ceph-mon[42070]: pgmap v140: 129 pgs: 129 active+clean; 474 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 428 op/s 2023-12-14T18:31:03.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:03 smithi097 ceph-mon[43486]: pgmap v140: 129 pgs: 129 active+clean; 474 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 428 op/s 2023-12-14T18:31:05.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:05 smithi160 ceph-mon[43840]: pgmap v141: 129 pgs: 129 active+clean; 474 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 428 op/s 2023-12-14T18:31:05.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:05 smithi093 ceph-mon[42070]: pgmap v141: 129 pgs: 129 active+clean; 474 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 428 op/s 2023-12-14T18:31:05.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:05 smithi097 ceph-mon[43486]: pgmap v141: 129 pgs: 129 active+clean; 474 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 428 op/s 2023-12-14T18:31:07.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:07 smithi160 ceph-mon[43840]: pgmap v142: 129 pgs: 129 active+clean; 483 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 585 op/s 2023-12-14T18:31:07.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:07 smithi093 ceph-mon[42070]: pgmap v142: 129 pgs: 129 active+clean; 483 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 585 op/s 2023-12-14T18:31:07.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:07 smithi097 ceph-mon[43486]: pgmap v142: 129 pgs: 129 active+clean; 483 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 MiB/s rd, 3.9 MiB/s wr, 585 op/s 2023-12-14T18:31:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.499+0000 7f2e96f27700 1 -- 172.21.15.93:0/2971582015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e90106920 msgr2=0x7f2e90106d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.499+0000 7f2e96f27700 1 --2- 172.21.15.93:0/2971582015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e90106920 0x7f2e90106d90 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c009990 tx=0x7f2e7c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.499+0000 7f2e96f27700 1 -- 172.21.15.93:0/2971582015 shutdown_connections 2023-12-14T18:31:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.499+0000 7f2e96f27700 1 --2- 172.21.15.93:0/2971582015 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e901072d0 0x7f2e901136b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.499+0000 7f2e96f27700 1 --2- 172.21.15.93:0/2971582015 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9010c880 0x7f2e9010cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.499+0000 7f2e96f27700 1 --2- 172.21.15.93:0/2971582015 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e90106920 0x7f2e90106d90 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.500 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.499+0000 7f2e96f27700 1 -- 172.21.15.93:0/2971582015 >> 172.21.15.93:0/2971582015 conn(0x7f2e90074840 msgr2=0x7f2e90076c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.499+0000 7f2e96f27700 1 -- 172.21.15.93:0/2971582015 shutdown_connections 2023-12-14T18:31:08.501 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.499+0000 7f2e96f27700 1 -- 172.21.15.93:0/2971582015 wait complete. 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e96f27700 1 Processor -- start 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e96f27700 1 -- start start 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e96f27700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90106920 0x7f2e901a26e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e96f27700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e901072d0 0x7f2e901a2c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e96f27700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9010c880 0x7f2e9019c750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e96f27700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9019cdc0 con 0x7f2e901072d0 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e96f27700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9019cf30 con 0x7f2e90106920 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e96f27700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e9019d0f0 con 0x7f2e9010c880 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e94cc3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90106920 0x7f2e901a26e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.502 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e8ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e901072d0 0x7f2e901a2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e94cc3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90106920 0x7f2e901a26e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:48580/0 (socket says 172.21.15.93:48580) 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e8ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e901072d0 0x7f2e901a2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49184/0 (socket says 172.21.15.93:49184) 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e94cc3700 1 -- 172.21.15.93:0/3829727651 learned_addr learned my addr 172.21.15.93:0/3829727651 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e954c4700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9010c880 0x7f2e9019c750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.500+0000 7f2e94cc3700 1 -- 172.21.15.93:0/3829727651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9010c880 msgr2=0x7f2e9019c750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9010c880 0x7f2e9019c750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e94cc3700 1 -- 172.21.15.93:0/3829727651 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e901072d0 msgr2=0x7f2e901a2c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e901072d0 0x7f2e901a2c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e94cc3700 1 -- 172.21.15.93:0/3829727651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e7c0095f0 con 0x7f2e90106920 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90106920 0x7f2e901a26e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400e970 tx=0x7f2e8400ec80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e8dffb700 1 -- 172.21.15.93:0/3829727651 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84009790 con 0x7f2e90106920 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e96f27700 1 -- 172.21.15.93:0/3829727651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e90116960 con 0x7f2e90106920 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e8dffb700 1 -- 172.21.15.93:0/3829727651 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e8400c7e0 con 0x7f2e90106920 2023-12-14T18:31:08.503 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e8dffb700 1 -- 172.21.15.93:0/3829727651 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e84003cb0 con 0x7f2e90106920 2023-12-14T18:31:08.504 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.501+0000 7f2e96f27700 1 -- 172.21.15.93:0/3829727651 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e901167d0 con 0x7f2e90106920 2023-12-14T18:31:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.504+0000 7f2e8dffb700 1 -- 172.21.15.93:0/3829727651 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2e8400c950 con 0x7f2e90106920 2023-12-14T18:31:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.505+0000 7f2e8dffb700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f2e800a1380 0x7f2e800a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.505+0000 7f2e8dffb700 1 -- 172.21.15.93:0/3829727651 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f2e84014070 con 0x7f2e90106920 2023-12-14T18:31:08.506 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.505+0000 7f2e8ffff700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f2e800a1380 0x7f2e800a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.505+0000 7f2e8ffff700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f2e800a1380 0x7f2e800a3830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e9019dbc0 tx=0x7f2e7c005fd0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2ea30b5b80 1 Processor -- start 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2ea30b5b80 1 -- start start 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf48ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf49007e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf490d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf47ea580 con 0x555cf475d430 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4859810 con 0x555cf490b0f0 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4852ca0 con 0x555cf4900370 2023-12-14T18:31:08.507 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2e954c4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf490d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2e8ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf49007e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2e954c4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf490d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:48586/0 (socket says 172.21.15.93:48586) 2023-12-14T18:31:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2e954c4700 1 -- 172.21.15.93:0/1612191092 learned_addr learned my addr 172.21.15.93:0/1612191092 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:08.508 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.506+0000 7f2e8ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf49007e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:45556/0 (socket says 172.21.15.93:45556) 2023-12-14T18:31:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.507+0000 7f2e8ffff700 1 -- 172.21.15.93:0/1612191092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 msgr2=0x555cf490d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.507+0000 7f2e8ffff700 1 --2- 172.21.15.93:0/1612191092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf490d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.507+0000 7f2e8ffff700 1 -- 172.21.15.93:0/1612191092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 msgr2=0x555cf48ffe30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:31:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.507+0000 7f2e8ffff700 1 --2- 172.21.15.93:0/1612191092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf48ffe30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.507+0000 7f2e8ffff700 1 -- 172.21.15.93:0/1612191092 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf46d4350 con 0x555cf4900370 2023-12-14T18:31:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.507+0000 7f2e8ffff700 1 --2- 172.21.15.93:0/1612191092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf49007e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0098e0 tx=0x7f2e7c020000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35c6740518bcd2e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.508+0000 7f2e7affd700 1 -- 172.21.15.93:0/1612191092 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c02a070 con 0x555cf4900370 2023-12-14T18:31:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.508+0000 7f2e7affd700 1 -- 172.21.15.93:0/1612191092 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e7c01e040 con 0x555cf4900370 2023-12-14T18:31:08.509 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.508+0000 7f2e7affd700 1 -- 172.21.15.93:0/1612191092 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e7c00f6c0 con 0x555cf4900370 2023-12-14T18:31:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.509+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/1612191092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 msgr2=0x555cf49007e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.509+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/1612191092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf49007e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2e7c0098e0 tx=0x7f2e7c020000 comp rx=0 tx=0).stop 2023-12-14T18:31:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.509+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/1612191092 shutdown_connections 2023-12-14T18:31:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.509+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/1612191092 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf48ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.509+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/1612191092 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf490d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.509+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/1612191092 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf49007e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.509+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/1612191092 >> 172.21.15.93:0/1612191092 conn(0x555cf48f6890 msgr2=0x555cf48fc200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:08.510 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.509+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/1612191092 shutdown_connections 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.509+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/1612191092 wait complete. 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.510+0000 7f2ea30b5b80 1 -- start start 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.510+0000 7f2ea30b5b80 1 Processor -- start 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.510+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf475d430 0x555cf4907ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.510+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf4900370 0x555cf4908420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.510+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf490b0f0 0x555cf4908960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.510+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf47eef30 con 0x555cf490b0f0 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.510+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf47eebd0 con 0x555cf4900370 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.510+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf46d6640 con 0x555cf475d430 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.510+0000 7f2ea30b5b80 10 client.0 _ll_register_callbacks cb 0x555cf48128b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.511+0000 7f2e954c4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf490b0f0 0x555cf4908960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.511+0000 7f2e954c4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf490b0f0 0x555cf4908960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49190/0 (socket says 172.21.15.93:49190) 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.511+0000 7f2e954c4700 1 -- 172.21.15.93:0/2013643839 learned_addr learned my addr 172.21.15.93:0/2013643839 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:08.514 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.511+0000 7f2e954c4700 1 -- 172.21.15.93:0/2013643839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf475d430 msgr2=0x555cf4907ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.511+0000 7f2e954c4700 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf475d430 0x555cf4907ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.511+0000 7f2e954c4700 1 -- 172.21.15.93:0/2013643839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf4900370 msgr2=0x555cf4908420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.511+0000 7f2e954c4700 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf4900370 0x555cf4908420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.511+0000 7f2e954c4700 1 -- 172.21.15.93:0/2013643839 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e90113fb0 con 0x555cf490b0f0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2e954c4700 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf490b0f0 0x555cf4908960 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2e88012f60 tx=0x7f2e88012f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8462b570c763575b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2e6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2013643839 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800e040 con 0x555cf490b0f0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2013643839 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f2e8801a070 con 0x555cf490b0f0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2013643839 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x555cf48f3ea0 con 0x555cf490b0f0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2013643839 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e880092d0 con 0x555cf490b0f0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2ea30b5b80 10 client.15864 fetch_fsmap learned FSMap version 30 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2013643839 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555cf4819110 con 0x555cf490b0f0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2013643839 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x555cf48f3ea0 con 0x555cf490b0f0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2013643839 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f2e88004710 con 0x555cf490b0f0 2023-12-14T18:31:08.515 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2ea30b5b80 10 client.15864 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:31:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.512+0000 7f2ea30b5b80 10 client.15864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:31:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.513+0000 7f2ea30b5b80 20 client.15864 populate_metadata read hostname 'smithi093' 2023-12-14T18:31:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.513+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555cf493ccd0 0x555cf493f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.513+0000 7f2ea30b5b80 4 client.15864 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.513+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2013643839 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x555cf484bc80 con 0x555cf493ccd0 2023-12-14T18:31:08.516 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.513+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555cf493ccd0 0x555cf493f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.513+0000 7f2e6ffff700 10 client.15864 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.513+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555cf493ccd0 0x555cf493f0e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57cd25b25b989dc2 server_cookie=dc1efd05fc72a803 in_seq=0 out_seq=0 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.514+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2013643839 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x555cf484bc80 con 0x555cf493ccd0 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.514+0000 7f2e6ffff700 10 client.15864 handle_command_reply: tid=0 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.514+0000 7f2ea30b5b80 1 client.15864 shutdown 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.514+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2013643839 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf490b0f0 msgr2=0x555cf4908960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.514+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf490b0f0 0x555cf4908960 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f2e88012f60 tx=0x7f2e88012f90 comp rx=0 tx=0).stop 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2013643839 shutdown_connections 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555cf493ccd0 0x555cf493f0e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf490b0f0 0x555cf4908960 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf475d430 0x555cf4907ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2013643839 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf4900370 0x555cf4908420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2013643839 >> 172.21.15.93:0/2013643839 conn(0x555cf48f6890 msgr2=0x555cf48fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2e6ffff700 0 client.15864 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2013643839 shutdown_connections 2023-12-14T18:31:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2013643839 wait complete. 2023-12-14T18:31:08.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.515+0000 7f2ea30b5b80 20 client.15864 trim_cache size 0 max 16384 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.533+0000 7f2ea30b5b80 1 Processor -- start 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.533+0000 7f2ea30b5b80 1 -- start start 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf490a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf48ffe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf4911c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4900fb0 con 0x555cf475d430 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf48fafb0 con 0x555cf490b0f0 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf4903d70 con 0x555cf4900370 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2e8ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf48ffe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2e8ffff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf48ffe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:45566/0 (socket says 172.21.15.93:45566) 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2e8ffff700 1 -- 172.21.15.93:0/900995623 learned_addr learned my addr 172.21.15.93:0/900995623 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:08.535 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2e954c4700 1 --2- 172.21.15.93:0/900995623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf4911c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.534+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/900995623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf490a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.535+0000 7f2e954c4700 1 -- 172.21.15.93:0/900995623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 msgr2=0x555cf48ffe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.535+0000 7f2e954c4700 1 --2- 172.21.15.93:0/900995623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf48ffe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.536 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.535+0000 7f2e954c4700 1 -- 172.21.15.93:0/900995623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 msgr2=0x555cf490a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.535+0000 7f2e954c4700 1 --2- 172.21.15.93:0/900995623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf490a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.535+0000 7f2e954c4700 1 -- 172.21.15.93:0/900995623 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x555cf4818e60 con 0x555cf490b0f0 2023-12-14T18:31:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.535+0000 7f2e954c4700 1 --2- 172.21.15.93:0/900995623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf4911c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800b010 tx=0x7f2e88009e40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4293cb7fdfdc5c3e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.535+0000 7f2e7b7fe700 1 -- 172.21.15.93:0/900995623 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8800e070 con 0x555cf490b0f0 2023-12-14T18:31:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.535+0000 7f2e7b7fe700 1 -- 172.21.15.93:0/900995623 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e8800f040 con 0x555cf490b0f0 2023-12-14T18:31:08.537 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.535+0000 7f2e7b7fe700 1 -- 172.21.15.93:0/900995623 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e880093f0 con 0x555cf490b0f0 2023-12-14T18:31:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.536+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/900995623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 msgr2=0x555cf4911c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.536+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/900995623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf4911c20 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2e8800b010 tx=0x7f2e88009e40 comp rx=0 tx=0).stop 2023-12-14T18:31:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.536+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/900995623 shutdown_connections 2023-12-14T18:31:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.536+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/900995623 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf490a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.538 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.536+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/900995623 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf490b0f0 0x555cf4911c20 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.536+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/900995623 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf4900370 0x555cf48ffe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.536+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/900995623 >> 172.21.15.93:0/900995623 conn(0x555cf48f6890 msgr2=0x555cf4903390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.536+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/900995623 shutdown_connections 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.537+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/900995623 wait complete. 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.537+0000 7f2ea30b5b80 1 -- start start 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.537+0000 7f2ea30b5b80 1 Processor -- start 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf490de10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf4900370 0x555cf490e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2ea30b5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf490b0f0 0x555cf4989b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf498a0d0 con 0x555cf475d430 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf498a240 con 0x555cf4900370 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2ea30b5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x555cf498a400 con 0x555cf490b0f0 2023-12-14T18:31:08.539 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2ea30b5b80 10 client.0 _ll_register_callbacks cb 0x7f2e88012890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2e954c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf490b0f0 0x555cf4989b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2e954c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf490b0f0 0x555cf4989b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:45572/0 (socket says 172.21.15.93:45572) 2023-12-14T18:31:08.540 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2e954c4700 1 -- 172.21.15.93:0/2700727774 learned_addr learned my addr 172.21.15.93:0/2700727774 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf490de10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2e94cc3700 1 -- 172.21.15.93:0/2700727774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf490b0f0 msgr2=0x555cf4989b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf490b0f0 0x555cf4989b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2e94cc3700 1 -- 172.21.15.93:0/2700727774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf4900370 msgr2=0x555cf490e350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf4900370 0x555cf490e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.538+0000 7f2e94cc3700 1 -- 172.21.15.93:0/2700727774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x555cf4819110 con 0x555cf475d430 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.539+0000 7f2e94cc3700 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf490de10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2e840ca5e0 tx=0x7f2e8400e8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5fcd2797acbcb7c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.539+0000 7f2e6ffff700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.539+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2700727774 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e8409df90 con 0x555cf475d430 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.539+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2700727774 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f2e840a23b0 con 0x555cf475d430 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.539+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2700727774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f2e7c01e040 con 0x555cf475d430 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.539+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2700727774 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2e8400cc30 con 0x555cf475d430 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.539+0000 7f2ea30b5b80 10 client.15870 fetch_fsmap learned FSMap version 30 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.539+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2700727774 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x555cf47ff940 con 0x555cf475d430 2023-12-14T18:31:08.541 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.539+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2700727774 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2e7c01e040 con 0x555cf475d430 2023-12-14T18:31:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.540+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2700727774 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f2e8409a2d0 con 0x555cf475d430 2023-12-14T18:31:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.540+0000 7f2ea30b5b80 10 client.15870 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:31:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.540+0000 7f2ea30b5b80 10 client.15870 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:31:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.540+0000 7f2ea30b5b80 20 client.15870 populate_metadata read hostname 'smithi093' 2023-12-14T18:31:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.540+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555cf4991cb0 0x555cf49940c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.540+0000 7f2ea30b5b80 4 client.15870 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:31:08.542 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.540+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2700727774 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x555cf484b9a0 con 0x555cf4991cb0 2023-12-14T18:31:08.543 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.541+0000 7f2e8ffff700 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555cf4991cb0 0x555cf49940c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.543+0000 7f2e8ffff700 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555cf4991cb0 0x555cf49940c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86dae321fd8263b3 server_cookie=5177fda6737cdfcb in_seq=0 out_seq=0 2023-12-14T18:31:08.544 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:08.543+0000 7f2e6ffff700 10 client.15870 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:09.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:09 smithi160 ceph-mon[43840]: pgmap v143: 129 pgs: 129 active+clean; 483 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 442 op/s 2023-12-14T18:31:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.515+0000 7f2e6ffff700 1 -- 172.21.15.93:0/2700727774 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x555cf484b9a0 con 0x555cf4991cb0 2023-12-14T18:31:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.515+0000 7f2e6ffff700 10 client.15870 handle_command_reply: tid=0 2023-12-14T18:31:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.515+0000 7f2ea30b5b80 1 client.15870 shutdown 2023-12-14T18:31:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.515+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2700727774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 msgr2=0x555cf490de10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.515+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf490de10 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f2e840ca5e0 tx=0x7f2e8400e8f0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.516+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2700727774 shutdown_connections 2023-12-14T18:31:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.516+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x555cf4991cb0 0x555cf49940c0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.517 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.516+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x555cf475d430 0x555cf490de10 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.516+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x555cf490b0f0 0x555cf4989b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.516+0000 7f2e6ffff700 0 client.15870 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.516+0000 7f2ea30b5b80 1 --2- 172.21.15.93:0/2700727774 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x555cf4900370 0x555cf490e350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.516+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2700727774 >> 172.21.15.93:0/2700727774 conn(0x555cf48f6890 msgr2=0x555cf4901f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.516+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2700727774 shutdown_connections 2023-12-14T18:31:09.518 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.516+0000 7f2ea30b5b80 1 -- 172.21.15.93:0/2700727774 wait complete. 2023-12-14T18:31:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2ea30b5b80 20 client.15870 trim_cache size 0 max 16384 2023-12-14T18:31:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:31:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:31:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:31:09.519 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 -- 172.21.15.93:0/3829727651 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f2e800a1380 msgr2=0x7f2e800a3830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f2e800a1380 0x7f2e800a3830 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2e9019dbc0 tx=0x7f2e7c005fd0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 -- 172.21.15.93:0/3829727651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90106920 msgr2=0x7f2e901a26e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90106920 0x7f2e901a26e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2e8400e970 tx=0x7f2e8400ec80 comp rx=0 tx=0).stop 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 -- 172.21.15.93:0/3829727651 shutdown_connections 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2e9010c880 0x7f2e9019c750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2e90106920 0x7f2e901a26e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f2e800a1380 0x7f2e800a3830 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 --2- 172.21.15.93:0/3829727651 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2e901072d0 0x7f2e901a2c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:09.520 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 -- 172.21.15.93:0/3829727651 >> 172.21.15.93:0/3829727651 conn(0x7f2e90074840 msgr2=0x7f2e9010d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.517+0000 7f2e96f27700 1 -- 172.21.15.93:0/3829727651 shutdown_connections 2023-12-14T18:31:09.521 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:09.518+0000 7f2e96f27700 1 -- 172.21.15.93:0/3829727651 wait complete. 2023-12-14T18:31:09.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:09 smithi093 ceph-mon[42070]: pgmap v143: 129 pgs: 129 active+clean; 483 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 442 op/s 2023-12-14T18:31:09.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:09 smithi097 ceph-mon[43486]: pgmap v143: 129 pgs: 129 active+clean; 483 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 MiB/s rd, 2.2 MiB/s wr, 442 op/s 2023-12-14T18:31:11.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:11 smithi160 ceph-mon[43840]: pgmap v144: 129 pgs: 129 active+clean; 506 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 710 op/s 2023-12-14T18:31:11.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:11 smithi093 ceph-mon[42070]: pgmap v144: 129 pgs: 129 active+clean; 506 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 710 op/s 2023-12-14T18:31:11.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:11 smithi097 ceph-mon[43486]: pgmap v144: 129 pgs: 129 active+clean; 506 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.2 MiB/s wr, 710 op/s 2023-12-14T18:31:13.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:13 smithi160 ceph-mon[43840]: pgmap v145: 129 pgs: 129 active+clean; 506 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2023-12-14T18:31:13.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:13 smithi093 ceph-mon[42070]: pgmap v145: 129 pgs: 129 active+clean; 506 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2023-12-14T18:31:13.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:13 smithi097 ceph-mon[43486]: pgmap v145: 129 pgs: 129 active+clean; 506 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2023-12-14T18:31:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:15 smithi160 ceph-mon[43840]: pgmap v146: 129 pgs: 129 active+clean; 506 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2023-12-14T18:31:15.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:15 smithi093 ceph-mon[42070]: pgmap v146: 129 pgs: 129 active+clean; 506 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2023-12-14T18:31:15.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:15 smithi097 ceph-mon[43486]: pgmap v146: 129 pgs: 129 active+clean; 506 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 423 op/s 2023-12-14T18:31:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:17 smithi160 ceph-mon[43840]: pgmap v147: 129 pgs: 129 active+clean; 513 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.4 MiB/s wr, 614 op/s 2023-12-14T18:31:17.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:17 smithi093 ceph-mon[42070]: pgmap v147: 129 pgs: 129 active+clean; 513 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.4 MiB/s wr, 614 op/s 2023-12-14T18:31:17.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:17 smithi097 ceph-mon[43486]: pgmap v147: 129 pgs: 129 active+clean; 513 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.4 MiB/s wr, 614 op/s 2023-12-14T18:31:19.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:19 smithi093 ceph-mon[42070]: pgmap v148: 129 pgs: 129 active+clean; 513 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 457 op/s 2023-12-14T18:31:19.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:19 smithi097 ceph-mon[43486]: pgmap v148: 129 pgs: 129 active+clean; 513 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 457 op/s 2023-12-14T18:31:19.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:19 smithi160 ceph-mon[43840]: pgmap v148: 129 pgs: 129 active+clean; 513 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.6 MiB/s wr, 457 op/s 2023-12-14T18:31:21.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:21 smithi093 ceph-mon[42070]: pgmap v149: 129 pgs: 129 active+clean; 531 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 596 op/s 2023-12-14T18:31:21.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:21 smithi097 ceph-mon[43486]: pgmap v149: 129 pgs: 129 active+clean; 531 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 596 op/s 2023-12-14T18:31:21.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:21 smithi160 ceph-mon[43840]: pgmap v149: 129 pgs: 129 active+clean; 531 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 MiB/s rd, 5.1 MiB/s wr, 596 op/s 2023-12-14T18:31:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.676+0000 7f9fffd3f700 1 -- 172.21.15.93:0/2847857411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ff8104410 msgr2=0x7f9ff810bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:22.680 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.676+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/2847857411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ff8104410 0x7f9ff810bfe0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f9ff000b0d0 tx=0x7f9ff000b3e0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.678+0000 7f9fffd3f700 1 -- 172.21.15.93:0/2847857411 shutdown_connections 2023-12-14T18:31:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.678+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/2847857411 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ff8104410 0x7f9ff810bfe0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.678+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/2847857411 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80ff3d0 0x7f9ff80ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.678+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/2847857411 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff80ffd70 0x7f9ff8103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.681 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.678+0000 7f9fffd3f700 1 -- 172.21.15.93:0/2847857411 >> 172.21.15.93:0/2847857411 conn(0x7f9ff8076230 msgr2=0x7f9ff8076630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:22.686 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.682+0000 7f9fffd3f700 1 -- 172.21.15.93:0/2847857411 shutdown_connections 2023-12-14T18:31:22.689 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.688+0000 7f9fffd3f700 1 -- 172.21.15.93:0/2847857411 wait complete. 2023-12-14T18:31:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.688+0000 7f9fffd3f700 1 Processor -- start 2023-12-14T18:31:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.688+0000 7f9fffd3f700 1 -- start start 2023-12-14T18:31:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.689+0000 7f9fffd3f700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80ff3d0 0x7f9ff81060a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.689+0000 7f9fffd3f700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff80ffd70 0x7f9ff81065e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.689+0000 7f9fffd3f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ff8104410 0x7f9ff8106b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.689+0000 7f9fffd3f700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff819a850 con 0x7f9ff8104410 2023-12-14T18:31:22.691 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.689+0000 7f9fffd3f700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff819a9c0 con 0x7f9ff80ff3d0 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.689+0000 7f9fffd3f700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ff819ab80 con 0x7f9ff80ffd70 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.689+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80ff3d0 0x7f9ff81060a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.689+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80ff3d0 0x7f9ff81060a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:48982/0 (socket says 172.21.15.93:48982) 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.689+0000 7f9ffdadb700 1 -- 172.21.15.93:0/676308984 learned_addr learned my addr 172.21.15.93:0/676308984 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.690+0000 7f9ffe2dc700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ff8104410 0x7f9ff8106b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.690+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff80ffd70 0x7f9ff81065e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.690+0000 7f9ffe2dc700 1 -- 172.21.15.93:0/676308984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff80ffd70 msgr2=0x7f9ff81065e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.690+0000 7f9ffe2dc700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff80ffd70 0x7f9ff81065e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.690+0000 7f9ffe2dc700 1 -- 172.21.15.93:0/676308984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80ff3d0 msgr2=0x7f9ff81060a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:22.692 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.690+0000 7f9ffe2dc700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80ff3d0 0x7f9ff81060a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.690+0000 7f9ffe2dc700 1 -- 172.21.15.93:0/676308984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ff0009d00 con 0x7f9ff8104410 2023-12-14T18:31:22.693 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.691+0000 7f9ffe2dc700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ff8104410 0x7f9ff8106b20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009fd0 tx=0x7f9ff0007b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.693+0000 7f9feeffd700 1 -- 172.21.15.93:0/676308984 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff000e040 con 0x7f9ff8104410 2023-12-14T18:31:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.693+0000 7f9feeffd700 1 -- 172.21.15.93:0/676308984 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9ff001dd50 con 0x7f9ff8104410 2023-12-14T18:31:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.693+0000 7f9feeffd700 1 -- 172.21.15.93:0/676308984 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ff001be20 con 0x7f9ff8104410 2023-12-14T18:31:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.694+0000 7f9fffd3f700 1 -- 172.21.15.93:0/676308984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ff810f300 con 0x7f9ff8104410 2023-12-14T18:31:22.696 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.694+0000 7f9fffd3f700 1 -- 172.21.15.93:0/676308984 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ff810f170 con 0x7f9ff8104410 2023-12-14T18:31:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.698+0000 7fa00becdb80 1 Processor -- start 2023-12-14T18:31:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.700+0000 7f9feeffd700 1 -- 172.21.15.93:0/676308984 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9ff001b4e0 con 0x7f9ff8104410 2023-12-14T18:31:22.702 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.700+0000 7fa00becdb80 1 -- start start 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.700+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916b2f430 0x561916cd1e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.700+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916cd27e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.700+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 0x561916cf04e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.701+0000 7fa00becdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916bbc580 con 0x561916b2f430 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.701+0000 7fa00becdb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916c2b810 con 0x561916cd2370 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.701+0000 7fa00becdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916c24ca0 con 0x561916cee100 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.701+0000 7f9feeffd700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9fe40a1410 0x7f9fe40a38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.701+0000 7f9feeffd700 1 -- 172.21.15.93:0/676308984 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f9ff00cee10 con 0x7f9ff8104410 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.701+0000 7f9ffe2dc700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9fe40a1410 0x7f9fe40a38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.701+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916cd27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.704 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.701+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916cd27e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:48996/0 (socket says 172.21.15.93:48996) 2023-12-14T18:31:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.701+0000 7f9ffdadb700 1 -- 172.21.15.93:0/3285828402 learned_addr learned my addr 172.21.15.93:0/3285828402 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.702+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/3285828402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 0x561916cf04e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.702+0000 7f9ffdadb700 1 -- 172.21.15.93:0/3285828402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 msgr2=0x561916cf04e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.702+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/3285828402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 0x561916cf04e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.702+0000 7f9ffdadb700 1 -- 172.21.15.93:0/3285828402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916b2f430 msgr2=0x561916cd1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.702+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/3285828402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916b2f430 0x561916cd1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.702+0000 7f9ffdadb700 1 -- 172.21.15.93:0/3285828402 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561916aa6350 con 0x561916cd2370 2023-12-14T18:31:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.702+0000 7f9ffe2dc700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9fe40a1410 0x7f9fe40a38c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00986f0 tx=0x7f9ff001b880 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:22.705 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.703+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/3285828402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916cd27e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009bb0 tx=0x7f9fe8009ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8470a8bd6eec12ac server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.704+0000 7f9fe3fff700 1 -- 172.21.15.93:0/3285828402 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80096b0 con 0x561916cd2370 2023-12-14T18:31:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.704+0000 7f9fe3fff700 1 -- 172.21.15.93:0/3285828402 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe800ee50 con 0x561916cd2370 2023-12-14T18:31:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.704+0000 7f9fe3fff700 1 -- 172.21.15.93:0/3285828402 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80123d0 con 0x561916cd2370 2023-12-14T18:31:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.705+0000 7fa00becdb80 1 -- 172.21.15.93:0/3285828402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 msgr2=0x561916cd27e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.705+0000 7fa00becdb80 1 --2- 172.21.15.93:0/3285828402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916cd27e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8009bb0 tx=0x7f9fe8009ec0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.706+0000 7fa00becdb80 1 -- 172.21.15.93:0/3285828402 shutdown_connections 2023-12-14T18:31:22.707 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.706+0000 7fa00becdb80 1 --2- 172.21.15.93:0/3285828402 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916b2f430 0x561916cd1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.706+0000 7fa00becdb80 1 --2- 172.21.15.93:0/3285828402 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 0x561916cf04e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.706+0000 7fa00becdb80 1 --2- 172.21.15.93:0/3285828402 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916cd27e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.706+0000 7fa00becdb80 1 -- 172.21.15.93:0/3285828402 >> 172.21.15.93:0/3285828402 conn(0x561916cc8890 msgr2=0x561916cce200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.706+0000 7fa00becdb80 1 -- 172.21.15.93:0/3285828402 shutdown_connections 2023-12-14T18:31:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.706+0000 7fa00becdb80 1 -- 172.21.15.93:0/3285828402 wait complete. 2023-12-14T18:31:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.706+0000 7fa00becdb80 1 -- start start 2023-12-14T18:31:22.708 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.707+0000 7fa00becdb80 1 Processor -- start 2023-12-14T18:31:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.707+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916b2f430 0x561916cedb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.707+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916ceae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.707+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cee100 0x561916ceb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.707+0000 7fa00becdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916bc0f30 con 0x561916cee100 2023-12-14T18:31:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.707+0000 7fa00becdb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916bc0bd0 con 0x561916cd2370 2023-12-14T18:31:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.707+0000 7fa00becdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916aa8640 con 0x561916b2f430 2023-12-14T18:31:22.709 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.707+0000 7fa00becdb80 10 client.0 _ll_register_callbacks cb 0x561916be48b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:22.710 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.708+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916ceae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.709+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916ceae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:49012/0 (socket says 172.21.15.93:49012) 2023-12-14T18:31:22.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.709+0000 7f9ffdadb700 1 -- 172.21.15.93:0/127007816 learned_addr learned my addr 172.21.15.93:0/127007816 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.710+0000 7f9ffdadb700 1 -- 172.21.15.93:0/127007816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916b2f430 msgr2=0x561916cedb30 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:31:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.710+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916b2f430 0x561916cedb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.710+0000 7f9ffdadb700 1 -- 172.21.15.93:0/127007816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cee100 msgr2=0x561916ceb590 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-14T18:31:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.710+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cee100 0x561916ceb590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.710+0000 7f9ffdadb700 1 -- 172.21.15.93:0/127007816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fe8009810 con 0x561916cd2370 2023-12-14T18:31:22.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.711+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916ceae50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8013b70 tx=0x7f9fe80138f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=953ded54545b3c8e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.712+0000 7f9fe17fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-12-14T18:31:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.712+0000 7f9fe17fa700 1 -- 172.21.15.93:0/127007816 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8009690 con 0x561916cd2370 2023-12-14T18:31:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.712+0000 7f9fe17fa700 1 -- 172.21.15.93:0/127007816 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f9fe800eba0 con 0x561916cd2370 2023-12-14T18:31:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.712+0000 7fa00becdb80 1 -- 172.21.15.93:0/127007816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x561916cc5ea0 con 0x561916cd2370 2023-12-14T18:31:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.712+0000 7f9fe17fa700 1 -- 172.21.15.93:0/127007816 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe801c450 con 0x561916cd2370 2023-12-14T18:31:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.712+0000 7fa00becdb80 10 client.25499 fetch_fsmap learned FSMap version 30 2023-12-14T18:31:22.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.712+0000 7fa00becdb80 1 -- 172.21.15.93:0/127007816 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561916beb110 con 0x561916cd2370 2023-12-14T18:31:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.713+0000 7f9fe17fa700 1 -- 172.21.15.93:0/127007816 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x561916cc5ea0 con 0x561916cd2370 2023-12-14T18:31:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.713+0000 7f9fe17fa700 1 -- 172.21.15.93:0/127007816 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9fe802c070 con 0x561916cd2370 2023-12-14T18:31:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.713+0000 7fa00becdb80 10 client.25499 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:31:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.713+0000 7fa00becdb80 10 client.25499 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:31:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.713+0000 7fa00becdb80 20 client.25499 populate_metadata read hostname 'smithi093' 2023-12-14T18:31:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.713+0000 7fa00becdb80 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561916cfda80 0x561916cffe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.713+0000 7fa00becdb80 4 client.25499 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:22.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.713+0000 7fa00becdb80 1 -- 172.21.15.93:0/127007816 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x561916c1dc80 con 0x561916cfda80 2023-12-14T18:31:22.716 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.715+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561916cfda80 0x561916cffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.724+0000 7f9fe17fa700 10 client.25499 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:22.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.724+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561916cfda80 0x561916cffe90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f919923a1761fa1 server_cookie=dfea26870b69efd7 in_seq=0 out_seq=0 2023-12-14T18:31:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.730+0000 7f9fe17fa700 1 -- 172.21.15.93:0/127007816 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x561916c1dc80 con 0x561916cfda80 2023-12-14T18:31:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.730+0000 7f9fe17fa700 10 client.25499 handle_command_reply: tid=0 2023-12-14T18:31:22.732 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.730+0000 7fa00becdb80 1 client.25499 shutdown 2023-12-14T18:31:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.734+0000 7fa00becdb80 1 -- 172.21.15.93:0/127007816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 msgr2=0x561916ceae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.734+0000 7fa00becdb80 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916ceae50 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8013b70 tx=0x7f9fe80138f0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.734+0000 7fa00becdb80 1 -- 172.21.15.93:0/127007816 shutdown_connections 2023-12-14T18:31:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.734+0000 7fa00becdb80 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561916cfda80 0x561916cffe90 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.736 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.734+0000 7fa00becdb80 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cee100 0x561916ceb590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.734+0000 7fa00becdb80 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916b2f430 0x561916cedb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.734+0000 7fa00becdb80 1 --2- 172.21.15.93:0/127007816 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cd2370 0x561916ceae50 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.734+0000 7fa00becdb80 1 -- 172.21.15.93:0/127007816 >> 172.21.15.93:0/127007816 conn(0x561916cc8890 msgr2=0x561916ccdd90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.734+0000 7f9fe17fa700 0 client.25499 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.735+0000 7fa00becdb80 1 -- 172.21.15.93:0/127007816 shutdown_connections 2023-12-14T18:31:22.737 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.735+0000 7fa00becdb80 1 -- 172.21.15.93:0/127007816 wait complete. 2023-12-14T18:31:22.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.738+0000 7fa00becdb80 20 client.25499 trim_cache size 0 max 16384 2023-12-14T18:31:22.765 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.764+0000 7fa00becdb80 1 Processor -- start 2023-12-14T18:31:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7fa00becdb80 1 -- start start 2023-12-14T18:31:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916b2f430 0x561916cea6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cd2370 0x561916ccc930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 0x561916cf9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7fa00becdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916ceac30 con 0x561916cd2370 2023-12-14T18:31:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7fa00becdb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916ccce70 con 0x561916b2f430 2023-12-14T18:31:22.771 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7fa00becdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916cebb70 con 0x561916cee100 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cd2370 0x561916ccc930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cd2370 0x561916ccc930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:39130/0 (socket says 172.21.15.93:39130) 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7f9ffdadb700 1 -- 172.21.15.93:0/4095462436 learned_addr learned my addr 172.21.15.93:0/4095462436 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/4095462436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 0x561916cf9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7f9ffdadb700 1 -- 172.21.15.93:0/4095462436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 msgr2=0x561916cf9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/4095462436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 0x561916cf9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7f9ffdadb700 1 -- 172.21.15.93:0/4095462436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916b2f430 msgr2=0x561916cea6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/4095462436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916b2f430 0x561916cea6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.769+0000 7f9ffdadb700 1 -- 172.21.15.93:0/4095462436 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x561916beae60 con 0x561916cd2370 2023-12-14T18:31:22.773 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.770+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/4095462436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 0x561916cf9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:31:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.770+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/4095462436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cd2370 0x561916ccc930 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8006f60 tx=0x7f9fe800ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e53be6a26a594e19 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.770+0000 7f9fecff9700 1 -- 172.21.15.93:0/4095462436 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe8009690 con 0x561916cd2370 2023-12-14T18:31:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.770+0000 7f9fecff9700 1 -- 172.21.15.93:0/4095462436 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe8024990 con 0x561916cd2370 2023-12-14T18:31:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.770+0000 7f9fecff9700 1 -- 172.21.15.93:0/4095462436 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe80123d0 con 0x561916cd2370 2023-12-14T18:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.771+0000 7fa00becdb80 1 -- 172.21.15.93:0/4095462436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cd2370 msgr2=0x561916ccc930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.771+0000 7fa00becdb80 1 --2- 172.21.15.93:0/4095462436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cd2370 0x561916ccc930 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8006f60 tx=0x7f9fe800ffd0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.773+0000 7fa00becdb80 1 -- 172.21.15.93:0/4095462436 shutdown_connections 2023-12-14T18:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.773+0000 7fa00becdb80 1 --2- 172.21.15.93:0/4095462436 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916cee100 0x561916cf9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.773+0000 7fa00becdb80 1 --2- 172.21.15.93:0/4095462436 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916b2f430 0x561916cea6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.773+0000 7fa00becdb80 1 --2- 172.21.15.93:0/4095462436 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cd2370 0x561916ccc930 secure :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f9fe8006f60 tx=0x7f9fe800ffd0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.773+0000 7fa00becdb80 1 -- 172.21.15.93:0/4095462436 >> 172.21.15.93:0/4095462436 conn(0x561916cc8890 msgr2=0x561916d047b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.776+0000 7fa00becdb80 1 -- 172.21.15.93:0/4095462436 shutdown_connections 2023-12-14T18:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.776+0000 7fa00becdb80 1 -- 172.21.15.93:0/4095462436 wait complete. 2023-12-14T18:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.777+0000 7fa00becdb80 1 -- start start 2023-12-14T18:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.777+0000 7fa00becdb80 1 Processor -- start 2023-12-14T18:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.778+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916b2f430 0x561916cf2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.778+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cee100 0x561916cf31f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.778+0000 7fa00becdb80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cf3730 0x561916d50900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.778+0000 7fa00becdb80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916cf51f0 con 0x561916cf3730 2023-12-14T18:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.778+0000 7fa00becdb80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916d50e40 con 0x561916cee100 2023-12-14T18:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.778+0000 7fa00becdb80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x561916d50f80 con 0x561916b2f430 2023-12-14T18:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.778+0000 7fa00becdb80 10 client.0 _ll_register_callbacks cb 0x7f9ff400a350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.778+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cee100 0x561916cf31f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffdadb700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cee100 0x561916cf31f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:49030/0 (socket says 172.21.15.93:49030) 2023-12-14T18:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffdadb700 1 -- 172.21.15.93:0/2581701390 learned_addr learned my addr 172.21.15.93:0/2581701390 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916b2f430 0x561916cf2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cf3730 0x561916d50900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffdadb700 1 -- 172.21.15.93:0/2581701390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916b2f430 msgr2=0x561916cf2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916b2f430 0x561916cf2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffdadb700 1 -- 172.21.15.93:0/2581701390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cf3730 msgr2=0x561916d50900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cf3730 0x561916d50900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:22.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffdadb700 1 -- 172.21.15.93:0/2581701390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x561916beb110 con 0x561916cee100 2023-12-14T18:31:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9ffdadb700 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cee100 0x561916cf31f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80048a0 tx=0x7f9fe800edd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=455536eda5beb6de server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.779+0000 7f9fe17fa700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-12-14T18:31:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.780+0000 7f9fe17fa700 1 -- 172.21.15.93:0/2581701390 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fe803d070 con 0x561916cee100 2023-12-14T18:31:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.780+0000 7f9fe17fa700 1 -- 172.21.15.93:0/2581701390 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f9fe8000e40 con 0x561916cee100 2023-12-14T18:31:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.780+0000 7fa00becdb80 1 -- 172.21.15.93:0/2581701390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f9fe800ee50 con 0x561916cee100 2023-12-14T18:31:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.780+0000 7f9fe17fa700 1 -- 172.21.15.93:0/2581701390 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9fe8013800 con 0x561916cee100 2023-12-14T18:31:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.780+0000 7fa00becdb80 10 client.25502 fetch_fsmap learned FSMap version 30 2023-12-14T18:31:22.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.780+0000 7fa00becdb80 1 -- 172.21.15.93:0/2581701390 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x561916bd1940 con 0x561916cee100 2023-12-14T18:31:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.782+0000 7f9fe17fa700 1 -- 172.21.15.93:0/2581701390 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9fe800ee50 con 0x561916cee100 2023-12-14T18:31:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.782+0000 7f9fe17fa700 1 -- 172.21.15.93:0/2581701390 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f9fe8040030 con 0x561916cee100 2023-12-14T18:31:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.783+0000 7fa00becdb80 10 client.25502 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:31:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.783+0000 7fa00becdb80 10 client.25502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:31:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.783+0000 7fa00becdb80 20 client.25502 populate_metadata read hostname 'smithi093' 2023-12-14T18:31:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.783+0000 7fa00becdb80 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561916d52040 0x561916d54420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.783+0000 7fa00becdb80 4 client.25502 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:31:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.783+0000 7fa00becdb80 1 -- 172.21.15.93:0/2581701390 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x561916c1d9a0 con 0x561916d52040 2023-12-14T18:31:22.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.783+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561916d52040 0x561916d54420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.784+0000 7f9ffd2da700 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561916d52040 0x561916d54420 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=192008fc27dc29ea server_cookie=2539a860c3d4b6e0 in_seq=0 out_seq=0 2023-12-14T18:31:22.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:22.784+0000 7f9fe17fa700 10 client.25502 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:23.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:23 smithi093 ceph-mon[42070]: pgmap v150: 129 pgs: 129 active+clean; 531 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 329 op/s 2023-12-14T18:31:23.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:23 smithi097 ceph-mon[43486]: pgmap v150: 129 pgs: 129 active+clean; 531 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 329 op/s 2023-12-14T18:31:23.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:23 smithi160 ceph-mon[43840]: pgmap v150: 129 pgs: 129 active+clean; 531 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 329 op/s 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7f9fe17fa700 1 -- 172.21.15.93:0/2581701390 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x561916c1d9a0 con 0x561916d52040 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7f9fe17fa700 10 client.25502 handle_command_reply: tid=0 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7fa00becdb80 1 client.25502 shutdown 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7fa00becdb80 1 -- 172.21.15.93:0/2581701390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cee100 msgr2=0x561916cf31f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7fa00becdb80 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cee100 0x561916cf31f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f9fe80048a0 tx=0x7f9fe800edd0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7fa00becdb80 1 -- 172.21.15.93:0/2581701390 shutdown_connections 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7fa00becdb80 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x561916d52040 0x561916d54420 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7fa00becdb80 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x561916cf3730 0x561916d50900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7f9fe17fa700 0 client.25502 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7fa00becdb80 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x561916b2f430 0x561916cf2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.708+0000 7fa00becdb80 1 --2- 172.21.15.93:0/2581701390 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x561916cee100 0x561916cf31f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.711 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7fa00becdb80 1 -- 172.21.15.93:0/2581701390 >> 172.21.15.93:0/2581701390 conn(0x561916cc8890 msgr2=0x561916cffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7fa00becdb80 1 -- 172.21.15.93:0/2581701390 shutdown_connections 2023-12-14T18:31:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7fa00becdb80 1 -- 172.21.15.93:0/2581701390 wait complete. 2023-12-14T18:31:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7fa00becdb80 20 client.25502 trim_cache size 0 max 16384 2023-12-14T18:31:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:31:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:31:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:31:24.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 -- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9fe40a1410 msgr2=0x7f9fe40a38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9fe40a1410 0x7f9fe40a38c0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9ff00986f0 tx=0x7f9ff001b880 comp rx=0 tx=0).stop 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 -- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ff8104410 msgr2=0x7f9ff8106b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ff8104410 0x7f9ff8106b20 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f9ff0009fd0 tx=0x7f9ff0007b60 comp rx=0 tx=0).stop 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 -- 172.21.15.93:0/676308984 shutdown_connections 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9fe40a1410 0x7f9fe40a38c0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9ff8104410 0x7f9ff8106b20 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9ff80ff3d0 0x7f9ff81060a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 --2- 172.21.15.93:0/676308984 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9ff80ffd70 0x7f9ff81065e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:24.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.709+0000 7f9fffd3f700 1 -- 172.21.15.93:0/676308984 >> 172.21.15.93:0/676308984 conn(0x7f9ff8076230 msgr2=0x7f9ff8100d00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.710+0000 7f9fffd3f700 1 -- 172.21.15.93:0/676308984 shutdown_connections 2023-12-14T18:31:24.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:24.710+0000 7f9fffd3f700 1 -- 172.21.15.93:0/676308984 wait complete. 2023-12-14T18:31:25.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:25 smithi093 ceph-mon[42070]: pgmap v151: 129 pgs: 129 active+clean; 531 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 329 op/s 2023-12-14T18:31:25.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:25 smithi097 ceph-mon[43486]: pgmap v151: 129 pgs: 129 active+clean; 531 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 329 op/s 2023-12-14T18:31:25.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:25 smithi160 ceph-mon[43840]: pgmap v151: 129 pgs: 129 active+clean; 531 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 3.1 MiB/s wr, 329 op/s 2023-12-14T18:31:27.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:27 smithi093 ceph-mon[42070]: pgmap v152: 129 pgs: 129 active+clean; 552 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 612 op/s 2023-12-14T18:31:27.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:27 smithi097 ceph-mon[43486]: pgmap v152: 129 pgs: 129 active+clean; 552 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 612 op/s 2023-12-14T18:31:27.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:27 smithi160 ceph-mon[43840]: pgmap v152: 129 pgs: 129 active+clean; 552 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 612 op/s 2023-12-14T18:31:29.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:29 smithi093 ceph-mon[42070]: pgmap v153: 129 pgs: 129 active+clean; 552 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 422 op/s 2023-12-14T18:31:29.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:29 smithi097 ceph-mon[43486]: pgmap v153: 129 pgs: 129 active+clean; 552 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 422 op/s 2023-12-14T18:31:29.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:29 smithi160 ceph-mon[43840]: pgmap v153: 129 pgs: 129 active+clean; 552 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 422 op/s 2023-12-14T18:31:31.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:31 smithi093 ceph-mon[42070]: pgmap v154: 129 pgs: 129 active+clean; 553 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 575 op/s 2023-12-14T18:31:31.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:31 smithi097 ceph-mon[43486]: pgmap v154: 129 pgs: 129 active+clean; 553 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 575 op/s 2023-12-14T18:31:31.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:31 smithi160 ceph-mon[43840]: pgmap v154: 129 pgs: 129 active+clean; 553 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 575 op/s 2023-12-14T18:31:33.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:33 smithi093 ceph-mon[42070]: pgmap v155: 129 pgs: 129 active+clean; 553 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 436 op/s 2023-12-14T18:31:33.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:33 smithi097 ceph-mon[43486]: pgmap v155: 129 pgs: 129 active+clean; 553 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 436 op/s 2023-12-14T18:31:33.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:33 smithi160 ceph-mon[43840]: pgmap v155: 129 pgs: 129 active+clean; 553 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 436 op/s 2023-12-14T18:31:35.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:35 smithi093 ceph-mon[42070]: pgmap v156: 129 pgs: 129 active+clean; 553 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 436 op/s 2023-12-14T18:31:35.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:35 smithi097 ceph-mon[43486]: pgmap v156: 129 pgs: 129 active+clean; 553 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 436 op/s 2023-12-14T18:31:35.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:35 smithi160 ceph-mon[43840]: pgmap v156: 129 pgs: 129 active+clean; 553 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.3 MiB/s wr, 436 op/s 2023-12-14T18:31:37.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:37 smithi093 ceph-mon[42070]: pgmap v157: 129 pgs: 129 active+clean; 574 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 596 op/s 2023-12-14T18:31:37.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:37 smithi097 ceph-mon[43486]: pgmap v157: 129 pgs: 129 active+clean; 574 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 596 op/s 2023-12-14T18:31:37.754 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:37 smithi160 ceph-mon[43840]: pgmap v157: 129 pgs: 129 active+clean; 574 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 5.1 MiB/s wr, 596 op/s 2023-12-14T18:31:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.856+0000 7fdef9808700 1 -- 172.21.15.93:0/2407489905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef40ff380 msgr2=0x7fdef4104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:37.858 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.856+0000 7fdef9808700 1 --2- 172.21.15.93:0/2407489905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef40ff380 0x7fdef4104e50 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7fdee80099c0 tx=0x7fdee8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.857+0000 7fdef9808700 1 -- 172.21.15.93:0/2407489905 shutdown_connections 2023-12-14T18:31:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.857+0000 7fdef9808700 1 --2- 172.21.15.93:0/2407489905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef4105580 0x7fdef410d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.857+0000 7fdef9808700 1 --2- 172.21.15.93:0/2407489905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdef40fe9e0 0x7fdef40fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.857+0000 7fdef9808700 1 --2- 172.21.15.93:0/2407489905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef40ff380 0x7fdef4104e50 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.857+0000 7fdef9808700 1 -- 172.21.15.93:0/2407489905 >> 172.21.15.93:0/2407489905 conn(0x7fdef4075810 msgr2=0x7fdef4075c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:37.859 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.857+0000 7fdef9808700 1 -- 172.21.15.93:0/2407489905 shutdown_connections 2023-12-14T18:31:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.857+0000 7fdef9808700 1 -- 172.21.15.93:0/2407489905 wait complete. 2023-12-14T18:31:37.860 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.857+0000 7fdef9808700 1 Processor -- start 2023-12-14T18:31:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef9808700 1 -- start start 2023-12-14T18:31:37.861 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef9808700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40fe9e0 0x7fdef40ff950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef9808700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdef40ff380 0x7fdef40ffe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef9808700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef4105580 0x7fdef41a4580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef9808700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41006c0 con 0x7fdef4105580 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef9808700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41a4ac0 con 0x7fdef40fe9e0 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef9808700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdef41a4c00 con 0x7fdef40ff380 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef4105580 0x7fdef41a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40fe9e0 0x7fdef40ff950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdef40ff380 0x7fdef40ffe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef4105580 0x7fdef41a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43330/0 (socket says 172.21.15.93:43330) 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef37fe700 1 -- 172.21.15.93:0/2179127046 learned_addr learned my addr 172.21.15.93:0/2179127046 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef2ffd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40fe9e0 0x7fdef40ff950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:53536/0 (socket says 172.21.15.93:53536) 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef27fc700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdef40ff380 0x7fdef40ffe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:38512/0 (socket says 172.21.15.93:38512) 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef37fe700 1 -- 172.21.15.93:0/2179127046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdef40ff380 msgr2=0x7fdef40ffe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:37.862 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef37fe700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdef40ff380 0x7fdef40ffe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef37fe700 1 -- 172.21.15.93:0/2179127046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40fe9e0 msgr2=0x7fdef40ff950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef37fe700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40fe9e0 0x7fdef40ff950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef37fe700 1 -- 172.21.15.93:0/2179127046 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdee80095f0 con 0x7fdef4105580 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.858+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40fe9e0 0x7fdef40ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.859+0000 7fdef37fe700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef4105580 0x7fdef41a4580 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00ef30 tx=0x7fdeec00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.859+0000 7fdef8806700 1 -- 172.21.15.93:0/2179127046 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec018070 con 0x7fdef4105580 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.859+0000 7fdef8806700 1 -- 172.21.15.93:0/2179127046 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeec00f040 con 0x7fdef4105580 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.859+0000 7fdef9808700 1 -- 172.21.15.93:0/2179127046 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdef41102b0 con 0x7fdef4105580 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.859+0000 7fdef8806700 1 -- 172.21.15.93:0/2179127046 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec00cd00 con 0x7fdef4105580 2023-12-14T18:31:37.863 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.859+0000 7fdef9808700 1 -- 172.21.15.93:0/2179127046 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef4110120 con 0x7fdef4105580 2023-12-14T18:31:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.861+0000 7fdef8806700 1 -- 172.21.15.93:0/2179127046 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdeec003a00 con 0x7fdef4105580 2023-12-14T18:31:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.862+0000 7fdef8806700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdedc0a12c0 0x7fdedc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.862+0000 7fdef8806700 1 -- 172.21.15.93:0/2179127046 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fdeec0ccac0 con 0x7fdef4105580 2023-12-14T18:31:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.862+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdedc0a12c0 0x7fdedc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.864 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.862+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdedc0a12c0 0x7fdedc0a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdee4005950 tx=0x7fdee400b410 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:37.866 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdf05996b80 1 Processor -- start 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdf05996b80 1 -- start start 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d77c430 0x56010d91ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d91f370 0x56010d91f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d92c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdf05996b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d809580 con 0x56010d92a0f0 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdf05996b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d878810 con 0x56010d77c430 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdf05996b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d871ca0 con 0x56010d91f370 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d92c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d92c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43332/0 (socket says 172.21.15.93:43332) 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdef37fe700 1 -- 172.21.15.93:0/2326577870 learned_addr learned my addr 172.21.15.93:0/2326577870 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.863+0000 7fdef27fc700 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d91f370 0x56010d91f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d77c430 0x56010d91ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.867 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fdef37fe700 1 -- 172.21.15.93:0/2326577870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d91f370 msgr2=0x56010d91f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fdef37fe700 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d91f370 0x56010d91f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fdef37fe700 1 -- 172.21.15.93:0/2326577870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d77c430 msgr2=0x56010d91ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fdef37fe700 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d77c430 0x56010d91ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fdef37fe700 1 -- 172.21.15.93:0/2326577870 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010d6f3350 con 0x56010d92a0f0 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d77c430 0x56010d91ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fdef37fe700 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d92c4d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0090a0 tx=0x7fdeec099470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3af9e3196b7271aa server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fded9ffb700 1 -- 172.21.15.93:0/2326577870 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec084040 con 0x56010d92a0f0 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fded9ffb700 1 -- 172.21.15.93:0/2326577870 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeec004ad0 con 0x56010d92a0f0 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.864+0000 7fded9ffb700 1 -- 172.21.15.93:0/2326577870 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec099df0 con 0x56010d92a0f0 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 -- 172.21.15.93:0/2326577870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 msgr2=0x56010d92c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d92c4d0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0090a0 tx=0x7fdeec099470 comp rx=0 tx=0).stop 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 -- 172.21.15.93:0/2326577870 shutdown_connections 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d92c4d0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.868 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d77c430 0x56010d91ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 --2- 172.21.15.93:0/2326577870 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d91f370 0x56010d91f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 -- 172.21.15.93:0/2326577870 >> 172.21.15.93:0/2326577870 conn(0x56010d915890 msgr2=0x56010d91b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 -- 172.21.15.93:0/2326577870 shutdown_connections 2023-12-14T18:31:37.869 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 -- 172.21.15.93:0/2326577870 wait complete. 2023-12-14T18:31:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 -- start start 2023-12-14T18:31:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.865+0000 7fdf05996b80 1 Processor -- start 2023-12-14T18:31:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 0x56010d929b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d926eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d9275f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdf05996b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d80df30 con 0x56010d92a0f0 2023-12-14T18:31:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdf05996b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d80dbd0 con 0x56010d91f370 2023-12-14T18:31:37.871 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdf05996b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d6f5640 con 0x56010d77c430 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdf05996b80 10 client.0 _ll_register_callbacks cb 0x56010d8318b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d9275f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d9275f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43340/0 (socket says 172.21.15.93:43340) 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 0x56010d929b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef37fe700 1 -- 172.21.15.93:0/3830996744 learned_addr learned my addr 172.21.15.93:0/3830996744 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef27fc700 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d926eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef37fe700 1 -- 172.21.15.93:0/3830996744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 msgr2=0x56010d929b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef37fe700 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 0x56010d929b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef37fe700 1 -- 172.21.15.93:0/3830996744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 msgr2=0x56010d926eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef37fe700 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d926eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef37fe700 1 -- 172.21.15.93:0/3830996744 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdef4110120 con 0x56010d92a0f0 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fdef37fe700 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d9275f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0ca520 tx=0x7fdeec0990b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a92f05d7fac040 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fded2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.866+0000 7fded2ffd700 1 -- 172.21.15.93:0/3830996744 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec084040 con 0x56010d92a0f0 2023-12-14T18:31:37.872 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.867+0000 7fded2ffd700 1 -- 172.21.15.93:0/3830996744 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fdeec09d960 con 0x56010d92a0f0 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.867+0000 7fdf05996b80 1 -- 172.21.15.93:0/3830996744 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56010d912ea0 con 0x56010d92a0f0 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.867+0000 7fded2ffd700 1 -- 172.21.15.93:0/3830996744 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdeec099930 con 0x56010d92a0f0 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.867+0000 7fdf05996b80 10 client.15894 fetch_fsmap learned FSMap version 30 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.867+0000 7fdf05996b80 1 -- 172.21.15.93:0/3830996744 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56010d838110 con 0x56010d92a0f0 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.867+0000 7fded2ffd700 1 -- 172.21.15.93:0/3830996744 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x56010d912ea0 con 0x56010d92a0f0 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.868+0000 7fded2ffd700 1 -- 172.21.15.93:0/3830996744 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdeec09dc50 con 0x56010d92a0f0 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.868+0000 7fdf05996b80 10 client.15894 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.868+0000 7fdf05996b80 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.868+0000 7fdf05996b80 20 client.15894 populate_metadata read hostname 'smithi093' 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.868+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56010d9396e0 0x56010d93baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.868+0000 7fdf05996b80 4 client.15894 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.868+0000 7fdf05996b80 1 -- 172.21.15.93:0/3830996744 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56010d86ac80 con 0x56010d9396e0 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.868+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56010d9396e0 0x56010d93baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.869+0000 7fded2ffd700 10 client.15894 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:37.873 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.869+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56010d9396e0 0x56010d93baf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17db40e41795c807 server_cookie=1323cae6d92d310f in_seq=0 out_seq=0 2023-12-14T18:31:37.874 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fded2ffd700 1 -- 172.21.15.93:0/3830996744 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x56010d86ac80 con 0x56010d9396e0 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fded2ffd700 10 client.15894 handle_command_reply: tid=0 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 client.15894 shutdown 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 -- 172.21.15.93:0/3830996744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 msgr2=0x56010d9275f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d9275f0 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fdeec0ca520 tx=0x7fdeec0990b0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 -- 172.21.15.93:0/3830996744 shutdown_connections 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56010d9396e0 0x56010d93baf0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d9275f0 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 0x56010d929b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3830996744 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d926eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 -- 172.21.15.93:0/3830996744 >> 172.21.15.93:0/3830996744 conn(0x56010d915890 msgr2=0x56010d91ad90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fded2ffd700 0 client.15894 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 -- 172.21.15.93:0/3830996744 shutdown_connections 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.872+0000 7fdf05996b80 1 -- 172.21.15.93:0/3830996744 wait complete. 2023-12-14T18:31:37.875 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.873+0000 7fdf05996b80 20 client.15894 trim_cache size 0 max 16384 2023-12-14T18:31:37.890 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdf05996b80 1 Processor -- start 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdf05996b80 1 -- start start 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 0x56010d9297c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d922390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d926750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdf05996b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d922d70 con 0x56010d92a0f0 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdf05996b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d936550 con 0x56010d91f370 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdf05996b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d921d10 con 0x56010d77c430 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d926750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d926750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43348/0 (socket says 172.21.15.93:43348) 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef2ffd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 0x56010d9297c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef37fe700 1 -- 172.21.15.93:0/3890010425 learned_addr learned my addr 172.21.15.93:0/3890010425 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef27fc700 1 --2- 172.21.15.93:0/3890010425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d922390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.891 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef37fe700 1 -- 172.21.15.93:0/3890010425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 msgr2=0x56010d9297c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef37fe700 1 --2- 172.21.15.93:0/3890010425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 0x56010d9297c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef37fe700 1 -- 172.21.15.93:0/3890010425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 msgr2=0x56010d922390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef37fe700 1 --2- 172.21.15.93:0/3890010425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d922390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef37fe700 1 -- 172.21.15.93:0/3890010425 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56010d837e60 con 0x56010d92a0f0 2023-12-14T18:31:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.888+0000 7fdef37fe700 1 --2- 172.21.15.93:0/3890010425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d926750 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdeec097c80 tx=0x7fdeec004d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360e886610005a3b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdeda7fc700 1 -- 172.21.15.93:0/3890010425 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec084070 con 0x56010d92a0f0 2023-12-14T18:31:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdeda7fc700 1 -- 172.21.15.93:0/3890010425 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeec09a4f0 con 0x56010d92a0f0 2023-12-14T18:31:37.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdeda7fc700 1 -- 172.21.15.93:0/3890010425 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeec0c6070 con 0x56010d92a0f0 2023-12-14T18:31:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdf05996b80 1 -- 172.21.15.93:0/3890010425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 msgr2=0x56010d926750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3890010425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d926750 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fdeec097c80 tx=0x7fdeec004d30 comp rx=0 tx=0).stop 2023-12-14T18:31:37.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdf05996b80 1 -- 172.21.15.93:0/3890010425 shutdown_connections 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3890010425 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d92a0f0 0x56010d926750 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3890010425 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d77c430 0x56010d9297c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdf05996b80 1 --2- 172.21.15.93:0/3890010425 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d922390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdf05996b80 1 -- 172.21.15.93:0/3890010425 >> 172.21.15.93:0/3890010425 conn(0x56010d915890 msgr2=0x56010d935e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.889+0000 7fdf05996b80 1 -- 172.21.15.93:0/3890010425 shutdown_connections 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 1 -- 172.21.15.93:0/3890010425 wait complete. 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 1 -- start start 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 1 Processor -- start 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d77c430 0x56010d92ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d92f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d92a0f0 0x56010d986570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d986ae0 con 0x56010d77c430 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d986c50 con 0x56010d91f370 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x56010d986e10 con 0x56010d92a0f0 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdf05996b80 10 client.0 _ll_register_callbacks cb 0x7fdeec099700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:37.896 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdef2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d77c430 0x56010d92ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdef2ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d77c430 0x56010d92ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:43354/0 (socket says 172.21.15.93:43354) 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdef27fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d92f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdef2ffd700 1 -- 172.21.15.93:0/504499495 learned_addr learned my addr 172.21.15.93:0/504499495 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.890+0000 7fdef37fe700 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d92a0f0 0x56010d986570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fdef2ffd700 1 -- 172.21.15.93:0/504499495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d92a0f0 msgr2=0x56010d986570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d92a0f0 0x56010d986570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fdef2ffd700 1 -- 172.21.15.93:0/504499495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 msgr2=0x56010d92f3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d92f3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fdef2ffd700 1 -- 172.21.15.93:0/504499495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56010d838110 con 0x56010d77c430 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fdef2ffd700 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d77c430 0x56010d92ee60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x56010d93c3d0 tx=0x7fdee4013070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b6611f246b0b5c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fded2ffd700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fded2ffd700 1 -- 172.21.15.93:0/504499495 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdee4026040 con 0x56010d77c430 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fded2ffd700 1 -- 172.21.15.93:0/504499495 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fdee4017070 con 0x56010d77c430 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fdf05996b80 1 -- 172.21.15.93:0/504499495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fdeec004ad0 con 0x56010d77c430 2023-12-14T18:31:37.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fded2ffd700 1 -- 172.21.15.93:0/504499495 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fdee4004830 con 0x56010d77c430 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fdf05996b80 10 client.15900 fetch_fsmap learned FSMap version 30 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fdf05996b80 1 -- 172.21.15.93:0/504499495 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x56010d81e940 con 0x56010d77c430 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.891+0000 7fded2ffd700 1 -- 172.21.15.93:0/504499495 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdeec004ad0 con 0x56010d77c430 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fded2ffd700 1 -- 172.21.15.93:0/504499495 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fdee402c030 con 0x56010d77c430 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fdf05996b80 10 client.15900 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fdf05996b80 10 client.15900 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fdf05996b80 20 client.15900 populate_metadata read hostname 'smithi093' 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fdf05996b80 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56010d98e6e0 0x56010d990af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fdf05996b80 4 client.15900 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fdf05996b80 1 -- 172.21.15.93:0/504499495 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x56010d86a9a0 con 0x56010d98e6e0 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fdef27fc700 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56010d98e6e0 0x56010d990af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fdef27fc700 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56010d98e6e0 0x56010d990af0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d0433f509cfd06c server_cookie=2cba78fcd9d12150 in_seq=0 out_seq=0 2023-12-14T18:31:37.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:37.892+0000 7fded2ffd700 10 client.15900 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:39.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:39 smithi093 ceph-mon[42070]: pgmap v158: 129 pgs: 129 active+clean; 574 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 312 op/s 2023-12-14T18:31:39.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:39 smithi097 ceph-mon[43486]: pgmap v158: 129 pgs: 129 active+clean; 574 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 312 op/s 2023-12-14T18:31:39.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:39 smithi160 ceph-mon[43840]: pgmap v158: 129 pgs: 129 active+clean; 574 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 312 op/s 2023-12-14T18:31:40.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.779+0000 7fded2ffd700 1 -- 172.21.15.93:0/504499495 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x56010d86a9a0 con 0x56010d98e6e0 2023-12-14T18:31:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.779+0000 7fded2ffd700 10 client.15900 handle_command_reply: tid=0 2023-12-14T18:31:40.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.779+0000 7fdf05996b80 1 client.15900 shutdown 2023-12-14T18:31:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fdf05996b80 1 -- 172.21.15.93:0/504499495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d77c430 msgr2=0x56010d92ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:40.782 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fdf05996b80 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d77c430 0x56010d92ee60 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x56010d93c3d0 tx=0x7fdee4013070 comp rx=0 tx=0).stop 2023-12-14T18:31:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fdf05996b80 1 -- 172.21.15.93:0/504499495 shutdown_connections 2023-12-14T18:31:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fdf05996b80 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x56010d98e6e0 0x56010d990af0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fdf05996b80 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x56010d77c430 0x56010d92ee60 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fdf05996b80 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x56010d92a0f0 0x56010d986570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:40.783 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fdf05996b80 1 --2- 172.21.15.93:0/504499495 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x56010d91f370 0x56010d92f3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fdf05996b80 1 -- 172.21.15.93:0/504499495 >> 172.21.15.93:0/504499495 conn(0x56010d915890 msgr2=0x56010d922390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fded2ffd700 0 client.15900 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.781+0000 7fdf05996b80 1 -- 172.21.15.93:0/504499495 shutdown_connections 2023-12-14T18:31:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.782+0000 7fdf05996b80 1 -- 172.21.15.93:0/504499495 wait complete. 2023-12-14T18:31:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.782+0000 7fdf05996b80 20 client.15900 trim_cache size 0 max 16384 2023-12-14T18:31:40.784 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:31:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:31:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:31:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:31:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 -- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdedc0a12c0 msgr2=0x7fdedc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:40.785 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdedc0a12c0 0x7fdedc0a3770 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fdee4005950 tx=0x7fdee400b410 comp rx=0 tx=0).stop 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 -- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef4105580 msgr2=0x7fdef41a4580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef4105580 0x7fdef41a4580 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fdeec00ef30 tx=0x7fdeec00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 -- 172.21.15.93:0/2179127046 shutdown_connections 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdedc0a12c0 0x7fdedc0a3770 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdef4105580 0x7fdef41a4580 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdef40fe9e0 0x7fdef40ff950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 --2- 172.21.15.93:0/2179127046 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdef40ff380 0x7fdef40ffe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 -- 172.21.15.93:0/2179127046 >> 172.21.15.93:0/2179127046 conn(0x7fdef4075810 msgr2=0x7fdef40fc420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 -- 172.21.15.93:0/2179127046 shutdown_connections 2023-12-14T18:31:40.786 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:40.783+0000 7fdef9808700 1 -- 172.21.15.93:0/2179127046 wait complete. 2023-12-14T18:31:41.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:41 smithi093 ceph-mon[42070]: pgmap v159: 129 pgs: 129 active+clean; 592 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 599 op/s 2023-12-14T18:31:41.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:41 smithi097 ceph-mon[43486]: pgmap v159: 129 pgs: 129 active+clean; 592 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 599 op/s 2023-12-14T18:31:41.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:41 smithi160 ceph-mon[43840]: pgmap v159: 129 pgs: 129 active+clean; 592 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.8 MiB/s wr, 599 op/s 2023-12-14T18:31:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:43 smithi097 ceph-mon[43486]: pgmap v160: 129 pgs: 129 active+clean; 592 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 446 op/s 2023-12-14T18:31:43.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:43 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:31:43.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:43 smithi160 ceph-mon[43840]: pgmap v160: 129 pgs: 129 active+clean; 592 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 446 op/s 2023-12-14T18:31:43.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:43 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:31:43.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:43 smithi093 ceph-mon[42070]: pgmap v160: 129 pgs: 129 active+clean; 592 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 446 op/s 2023-12-14T18:31:43.818 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:43 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:31:44.592 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:44 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:31:44.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:44 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:31:44.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:31:44.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:44 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:31:44.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:44 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:31:44.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:44 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:31:44.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:31:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:44 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:31:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:44 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:31:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:31:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:44 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:31:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:44 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:31:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:44 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:31:44.746 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:31:44.814 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:44 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:31:44.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:44 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:31:44.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:31:44.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:44 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:31:44.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:44 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:31:44.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:44 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:31:44.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:31:45.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:45 smithi160 ceph-mon[43840]: pgmap v161: 129 pgs: 129 active+clean; 592 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 446 op/s 2023-12-14T18:31:46.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:45 smithi093 ceph-mon[42070]: pgmap v161: 129 pgs: 129 active+clean; 592 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 446 op/s 2023-12-14T18:31:46.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:45 smithi097 ceph-mon[43486]: pgmap v161: 129 pgs: 129 active+clean; 592 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.3 MiB/s wr, 446 op/s 2023-12-14T18:31:47.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:47 smithi160 ceph-mon[43840]: pgmap v162: 129 pgs: 129 active+clean; 595 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 604 op/s 2023-12-14T18:31:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:47 smithi093 ceph-mon[42070]: pgmap v162: 129 pgs: 129 active+clean; 595 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 604 op/s 2023-12-14T18:31:48.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:47 smithi097 ceph-mon[43486]: pgmap v162: 129 pgs: 129 active+clean; 595 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.8 MiB/s rd, 4.9 MiB/s wr, 604 op/s 2023-12-14T18:31:49.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:49 smithi160 ceph-mon[43840]: pgmap v163: 129 pgs: 129 active+clean; 595 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 445 op/s 2023-12-14T18:31:50.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:49 smithi093 ceph-mon[42070]: pgmap v163: 129 pgs: 129 active+clean; 595 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 445 op/s 2023-12-14T18:31:50.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:49 smithi097 ceph-mon[43486]: pgmap v163: 129 pgs: 129 active+clean; 595 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 3.1 MiB/s wr, 445 op/s 2023-12-14T18:31:51.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:51 smithi160 ceph-mon[43840]: pgmap v164: 129 pgs: 129 active+clean; 608 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 555 op/s 2023-12-14T18:31:52.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:51 smithi093 ceph-mon[42070]: pgmap v164: 129 pgs: 129 active+clean; 608 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 555 op/s 2023-12-14T18:31:52.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:51 smithi097 ceph-mon[43486]: pgmap v164: 129 pgs: 129 active+clean; 608 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.3 MiB/s wr, 555 op/s 2023-12-14T18:31:53.901 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 -- 172.21.15.93:0/3907115680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a401015f0 msgr2=0x7f8a40104e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 --2- 172.21.15.93:0/3907115680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a401015f0 0x7f8a40104e80 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f8a300099c0 tx=0x7f8a30009cd0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 -- 172.21.15.93:0/3907115680 shutdown_connections 2023-12-14T18:31:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 --2- 172.21.15.93:0/3907115680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a40105560 0x7f8a4010d0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 --2- 172.21.15.93:0/3907115680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a40100c50 0x7f8a40101020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 --2- 172.21.15.93:0/3907115680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a401015f0 0x7f8a40104e80 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 -- 172.21.15.93:0/3907115680 >> 172.21.15.93:0/3907115680 conn(0x7f8a40077560 msgr2=0x7f8a40077960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:53.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 -- 172.21.15.93:0/3907115680 shutdown_connections 2023-12-14T18:31:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 -- 172.21.15.93:0/3907115680 wait complete. 2023-12-14T18:31:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 Processor -- start 2023-12-14T18:31:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.897+0000 7f8a47879700 1 -- start start 2023-12-14T18:31:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a47879700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40100c50 0x7f8a401985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a47879700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a401015f0 0x7f8a40198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a47879700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a40105560 0x7f8a4019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a47879700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4019d5b0 con 0x7f8a40100c50 2023-12-14T18:31:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a47879700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4019d720 con 0x7f8a40105560 2023-12-14T18:31:53.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a47879700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4019d8e0 con 0x7f8a401015f0 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45615700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40100c50 0x7f8a401985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45615700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40100c50 0x7f8a401985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56332/0 (socket says 172.21.15.93:56332) 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45615700 1 -- 172.21.15.93:0/1992881568 learned_addr learned my addr 172.21.15.93:0/1992881568 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a44e14700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a401015f0 0x7f8a40198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45e16700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a40105560 0x7f8a4019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45615700 1 -- 172.21.15.93:0/1992881568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a401015f0 msgr2=0x7f8a40198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45615700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a401015f0 0x7f8a40198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45615700 1 -- 172.21.15.93:0/1992881568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a40105560 msgr2=0x7f8a4019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45615700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a40105560 0x7f8a4019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45615700 1 -- 172.21.15.93:0/1992881568 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a300095f0 con 0x7f8a40100c50 2023-12-14T18:31:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a45615700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40100c50 0x7f8a401985d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800d950 tx=0x7f8a3800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a367fc700 1 -- 172.21.15.93:0/1992881568 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3800f890 con 0x7f8a40100c50 2023-12-14T18:31:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a367fc700 1 -- 172.21.15.93:0/1992881568 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a38010460 con 0x7f8a40100c50 2023-12-14T18:31:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a367fc700 1 -- 172.21.15.93:0/1992881568 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3800e610 con 0x7f8a40100c50 2023-12-14T18:31:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a47879700 1 -- 172.21.15.93:0/1992881568 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a40110290 con 0x7f8a40100c50 2023-12-14T18:31:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.898+0000 7f8a47879700 1 -- 172.21.15.93:0/1992881568 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a40110100 con 0x7f8a40100c50 2023-12-14T18:31:53.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.901+0000 7f8a367fc700 1 -- 172.21.15.93:0/1992881568 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f8a380108d0 con 0x7f8a40100c50 2023-12-14T18:31:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.901+0000 7f8a367fc700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8a2c0a12c0 0x7f8a2c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.901+0000 7f8a44e14700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8a2c0a12c0 0x7f8a2c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.901+0000 7f8a367fc700 1 -- 172.21.15.93:0/1992881568 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f8a380ccdf0 con 0x7f8a40100c50 2023-12-14T18:31:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.902+0000 7f8a44e14700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8a2c0a12c0 0x7f8a2c0a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8a300098e0 tx=0x7f8a30000bc0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a53a07b80 1 Processor -- start 2023-12-14T18:31:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a53a07b80 1 -- start start 2023-12-14T18:31:53.914 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7238e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 0x55c3c72397e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c72464d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c7123580 con 0x55c3c7096430 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c7192810 con 0x55c3c72440f0 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c718bca0 con 0x55c3c7239370 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a45615700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7238e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a45615700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7238e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56340/0 (socket says 172.21.15.93:56340) 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a45615700 1 -- 172.21.15.93:0/3302219689 learned_addr learned my addr 172.21.15.93:0/3302219689 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a44e14700 1 --2- 172.21.15.93:0/3302219689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 0x55c3c72397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a45e16700 1 --2- 172.21.15.93:0/3302219689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c72464d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.915 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a45615700 1 -- 172.21.15.93:0/3302219689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 msgr2=0x55c3c72397e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a45615700 1 --2- 172.21.15.93:0/3302219689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 0x55c3c72397e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a45615700 1 -- 172.21.15.93:0/3302219689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 msgr2=0x55c3c72464d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a45615700 1 --2- 172.21.15.93:0/3302219689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c72464d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.903+0000 7f8a45615700 1 -- 172.21.15.93:0/3302219689 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c700d350 con 0x55c3c7096430 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a45615700 1 --2- 172.21.15.93:0/3302219689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7238e30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8a380960d0 tx=0x7f8a38099c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee3b81ed50b3dda8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a2b7fe700 1 -- 172.21.15.93:0/3302219689 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a380962e0 con 0x55c3c7096430 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a2b7fe700 1 -- 172.21.15.93:0/3302219689 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a38096440 con 0x55c3c7096430 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a2b7fe700 1 -- 172.21.15.93:0/3302219689 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a380a1b90 con 0x55c3c7096430 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a53a07b80 1 -- 172.21.15.93:0/3302219689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 msgr2=0x55c3c7238e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/3302219689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7238e30 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f8a380960d0 tx=0x7f8a38099c50 comp rx=0 tx=0).stop 2023-12-14T18:31:53.916 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a53a07b80 1 -- 172.21.15.93:0/3302219689 shutdown_connections 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/3302219689 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7238e30 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/3302219689 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c72464d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/3302219689 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 0x55c3c72397e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.904+0000 7f8a53a07b80 1 -- 172.21.15.93:0/3302219689 >> 172.21.15.93:0/3302219689 conn(0x55c3c722f890 msgr2=0x55c3c7235200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 -- 172.21.15.93:0/3302219689 shutdown_connections 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 -- 172.21.15.93:0/3302219689 wait complete. 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 -- start start 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 Processor -- start 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7243ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c7239370 0x55c3c7240e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.917 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c72440f0 0x55c3c7241480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c7127f30 con 0x55c3c7096430 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c7127bd0 con 0x55c3c7239370 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c700f640 con 0x55c3c72440f0 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a53a07b80 10 client.0 _ll_register_callbacks cb 0x55c3c714b8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.905+0000 7f8a45615700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7243ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a45615700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7243ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56342/0 (socket says 172.21.15.93:56342) 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a45615700 1 -- 172.21.15.93:0/2143969414 learned_addr learned my addr 172.21.15.93:0/2143969414 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a44e14700 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c7239370 0x55c3c7240e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a45e16700 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c72440f0 0x55c3c7241480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a45615700 1 -- 172.21.15.93:0/2143969414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c72440f0 msgr2=0x55c3c7241480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:53.918 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a45615700 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c72440f0 0x55c3c7241480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a45615700 1 -- 172.21.15.93:0/2143969414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c7239370 msgr2=0x55c3c7240e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a45615700 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c7239370 0x55c3c7240e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a45615700 1 -- 172.21.15.93:0/2143969414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a40110100 con 0x55c3c7096430 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a45615700 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7243ae0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8a380a1af0 tx=0x7f8a3809a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df023a3a1a7439a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a28ff9700 1 -- 172.21.15.93:0/2143969414 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a380962e0 con 0x55c3c7096430 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a28ff9700 1 -- 172.21.15.93:0/2143969414 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f8a380990c0 con 0x55c3c7096430 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a53a07b80 1 -- 172.21.15.93:0/2143969414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c3c722cea0 con 0x55c3c7096430 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a28ff9700 1 -- 172.21.15.93:0/2143969414 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a380b23b0 con 0x55c3c7096430 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a53a07b80 10 client.15912 fetch_fsmap learned FSMap version 30 2023-12-14T18:31:53.919 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.906+0000 7f8a53a07b80 1 -- 172.21.15.93:0/2143969414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c3c7152110 con 0x55c3c7096430 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a28ff9700 1 -- 172.21.15.93:0/2143969414 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c3c722cea0 con 0x55c3c7096430 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a28ff9700 1 -- 172.21.15.93:0/2143969414 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f8a38098bc0 con 0x55c3c7096430 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a53a07b80 10 client.15912 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a53a07b80 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a53a07b80 20 client.15912 populate_metadata read hostname 'smithi093' 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c3c7253630 0x55c3c7255a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a53a07b80 4 client.15912 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a53a07b80 1 -- 172.21.15.93:0/2143969414 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c3c7184c80 con 0x55c3c7253630 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a44e14700 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c3c7253630 0x55c3c7255a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a28ff9700 10 client.15912 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:53.920 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.907+0000 7f8a44e14700 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c3c7253630 0x55c3c7255a10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0e8b6a638bf873b server_cookie=92b9410735d604b7 in_seq=0 out_seq=0 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.910+0000 7f8a28ff9700 1 -- 172.21.15.93:0/2143969414 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c3c7184c80 con 0x55c3c7253630 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a28ff9700 10 client.15912 handle_command_reply: tid=0 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 client.15912 shutdown 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 -- 172.21.15.93:0/2143969414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 msgr2=0x55c3c7243ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7243ae0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f8a380a1af0 tx=0x7f8a3809a060 comp rx=0 tx=0).stop 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 -- 172.21.15.93:0/2143969414 shutdown_connections 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c3c7253630 0x55c3c7255a10 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c7243ae0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c72440f0 0x55c3c7241480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/2143969414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c7239370 0x55c3c7240e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.921 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 -- 172.21.15.93:0/2143969414 >> 172.21.15.93:0/2143969414 conn(0x55c3c722f890 msgr2=0x55c3c7234d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a28ff9700 0 client.15912 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 -- 172.21.15.93:0/2143969414 shutdown_connections 2023-12-14T18:31:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 1 -- 172.21.15.93:0/2143969414 wait complete. 2023-12-14T18:31:53.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.911+0000 7f8a53a07b80 20 client.15912 trim_cache size 0 max 16384 2023-12-14T18:31:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a53a07b80 1 Processor -- start 2023-12-14T18:31:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a53a07b80 1 -- start start 2023-12-14T18:31:53.933 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7096430 0x55c3c724abd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7239370 0x55c3c7234d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c724f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c723cca0 con 0x55c3c7239370 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c724b110 con 0x55c3c72440f0 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c723bd10 con 0x55c3c7096430 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a44e14700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7239370 0x55c3c7234d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a44e14700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7239370 0x55c3c7234d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56352/0 (socket says 172.21.15.93:56352) 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a44e14700 1 -- 172.21.15.93:0/1186547695 learned_addr learned my addr 172.21.15.93:0/1186547695 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a45e16700 1 --2- 172.21.15.93:0/1186547695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c724f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a45615700 1 --2- 172.21.15.93:0/1186547695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7096430 0x55c3c724abd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.934 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a44e14700 1 -- 172.21.15.93:0/1186547695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7096430 msgr2=0x55c3c724abd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a44e14700 1 --2- 172.21.15.93:0/1186547695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7096430 0x55c3c724abd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a44e14700 1 -- 172.21.15.93:0/1186547695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 msgr2=0x55c3c724f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a44e14700 1 --2- 172.21.15.93:0/1186547695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c724f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a44e14700 1 -- 172.21.15.93:0/1186547695 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c3c7151e60 con 0x55c3c7239370 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.927+0000 7f8a44e14700 1 --2- 172.21.15.93:0/1186547695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7239370 0x55c3c7234d30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8a401019c0 tx=0x7f8a30022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f864aba550f31f44 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a2bfff700 1 -- 172.21.15.93:0/1186547695 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3002c040 con 0x55c3c7239370 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a2bfff700 1 -- 172.21.15.93:0/1186547695 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a3002d040 con 0x55c3c7239370 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a2bfff700 1 -- 172.21.15.93:0/1186547695 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3000fad0 con 0x55c3c7239370 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1186547695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7239370 msgr2=0x55c3c7234d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1186547695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7239370 0x55c3c7234d30 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f8a401019c0 tx=0x7f8a30022000 comp rx=0 tx=0).stop 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1186547695 shutdown_connections 2023-12-14T18:31:53.935 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1186547695 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c724f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1186547695 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7096430 0x55c3c724abd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1186547695 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7239370 0x55c3c7234d30 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1186547695 >> 172.21.15.93:0/1186547695 conn(0x55c3c722f890 msgr2=0x55c3c7239740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1186547695 shutdown_connections 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.928+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1186547695 wait complete. 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.929+0000 7f8a53a07b80 1 -- start start 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.929+0000 7f8a53a07b80 1 Processor -- start 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.929+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c724a810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.929+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 0x55c3c7248bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.929+0000 7f8a53a07b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c72490f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.929+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c7249630 con 0x55c3c7096430 2023-12-14T18:31:53.936 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.929+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c7249770 con 0x55c3c72440f0 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.929+0000 7f8a53a07b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c3c7249930 con 0x55c3c7239370 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.929+0000 7f8a53a07b80 10 client.0 _ll_register_callbacks cb 0x7f8a3c009f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45615700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c724a810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45615700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c724a810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:56354/0 (socket says 172.21.15.93:56354) 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45615700 1 -- 172.21.15.93:0/1866245694 learned_addr learned my addr 172.21.15.93:0/1866245694 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a44e14700 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 0x55c3c7248bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45e16700 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c72490f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45615700 1 -- 172.21.15.93:0/1866245694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 msgr2=0x55c3c7248bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45615700 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 0x55c3c7248bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45615700 1 -- 172.21.15.93:0/1866245694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 msgr2=0x55c3c72490f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:31:53.937 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45615700 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c72490f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45615700 1 -- 172.21.15.93:0/1866245694 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4010d900 con 0x55c3c7096430 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a45615700 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c724a810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8a38005950 tx=0x7f8a380a20e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a63823ccc699bd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a28ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a28ff9700 1 -- 172.21.15.93:0/1866245694 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a380962e0 con 0x55c3c7096430 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a28ff9700 1 -- 172.21.15.93:0/1866245694 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f8a38099bd0 con 0x55c3c7096430 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1866245694 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f8a38096440 con 0x55c3c7096430 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a28ff9700 1 -- 172.21.15.93:0/1866245694 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a380b23b0 con 0x55c3c7096430 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a53a07b80 10 client.15918 fetch_fsmap learned FSMap version 30 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1866245694 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c3c7138940 con 0x55c3c7096430 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.930+0000 7f8a28ff9700 1 -- 172.21.15.93:0/1866245694 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f8a38096440 con 0x55c3c7096430 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a28ff9700 1 -- 172.21.15.93:0/1866245694 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f8a380991a0 con 0x55c3c7096430 2023-12-14T18:31:53.938 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a53a07b80 10 client.15918 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:31:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a53a07b80 10 client.15918 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:31:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a53a07b80 20 client.15918 populate_metadata read hostname 'smithi093' 2023-12-14T18:31:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c3c72a84c0 0x55c3c72aa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:31:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a53a07b80 4 client.15918 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:31:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1866245694 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c3c71849a0 con 0x55c3c72a84c0 2023-12-14T18:31:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a44e14700 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c3c72a84c0 0x55c3c72aa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:31:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a44e14700 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c3c72a84c0 0x55c3c72aa8d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6882717df5c9fbde server_cookie=bc89446dbdb4e061 in_seq=0 out_seq=0 2023-12-14T18:31:53.939 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:53.931+0000 7f8a28ff9700 10 client.15918 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:53.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:53 smithi160 ceph-mon[43840]: pgmap v165: 129 pgs: 129 active+clean; 608 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2023-12-14T18:31:54.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:53 smithi093 ceph-mon[42070]: pgmap v165: 129 pgs: 129 active+clean; 608 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2023-12-14T18:31:54.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:53 smithi097 ceph-mon[43486]: pgmap v165: 129 pgs: 129 active+clean; 608 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2023-12-14T18:31:55.663 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.662+0000 7f8a28ff9700 1 -- 172.21.15.93:0/1866245694 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c3c71849a0 con 0x55c3c72a84c0 2023-12-14T18:31:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.662+0000 7f8a28ff9700 10 client.15918 handle_command_reply: tid=0 2023-12-14T18:31:55.664 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.662+0000 7f8a53a07b80 1 client.15918 shutdown 2023-12-14T18:31:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.662+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1866245694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 msgr2=0x55c3c724a810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:55.665 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.662+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c724a810 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f8a38005950 tx=0x7f8a380a20e0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.666+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1866245694 shutdown_connections 2023-12-14T18:31:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.666+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c3c72a84c0 0x55c3c72aa8d0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.668 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.666+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c3c7096430 0x55c3c724a810 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.666+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c3c72440f0 0x55c3c72490f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.666+0000 7f8a53a07b80 1 --2- 172.21.15.93:0/1866245694 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c3c7239370 0x55c3c7248bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.666+0000 7f8a28ff9700 0 client.15918 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:31:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.666+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1866245694 >> 172.21.15.93:0/1866245694 conn(0x55c3c722f890 msgr2=0x55c3c7255640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.666+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1866245694 shutdown_connections 2023-12-14T18:31:55.669 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.666+0000 7f8a53a07b80 1 -- 172.21.15.93:0/1866245694 wait complete. 2023-12-14T18:31:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.671+0000 7f8a53a07b80 20 client.15918 trim_cache size 0 max 16384 2023-12-14T18:31:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:31:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:31:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:31:55.672 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:31:55.673 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.672+0000 7f8a47879700 1 -- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8a2c0a12c0 msgr2=0x7f8a2c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.672+0000 7f8a47879700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8a2c0a12c0 0x7f8a2c0a3770 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8a300098e0 tx=0x7f8a30000bc0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.672+0000 7f8a47879700 1 -- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40100c50 msgr2=0x7f8a401985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:31:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.672+0000 7f8a47879700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40100c50 0x7f8a401985d0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f8a3800d950 tx=0x7f8a3800dc60 comp rx=0 tx=0).stop 2023-12-14T18:31:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.672+0000 7f8a47879700 1 -- 172.21.15.93:0/1992881568 shutdown_connections 2023-12-14T18:31:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.672+0000 7f8a47879700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f8a2c0a12c0 0x7f8a2c0a3770 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.672+0000 7f8a47879700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f8a40100c50 0x7f8a401985d0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.672+0000 7f8a47879700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f8a40105560 0x7f8a4019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.672+0000 7f8a47879700 1 --2- 172.21.15.93:0/1992881568 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f8a401015f0 0x7f8a40198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:31:55.674 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.673+0000 7f8a47879700 1 -- 172.21.15.93:0/1992881568 >> 172.21.15.93:0/1992881568 conn(0x7f8a40077560 msgr2=0x7f8a4010c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:31:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.673+0000 7f8a47879700 1 -- 172.21.15.93:0/1992881568 shutdown_connections 2023-12-14T18:31:55.675 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:31:55.673+0000 7f8a47879700 1 -- 172.21.15.93:0/1992881568 wait complete. 2023-12-14T18:31:55.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:55 smithi160 ceph-mon[43840]: pgmap v166: 129 pgs: 129 active+clean; 608 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2023-12-14T18:31:56.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:55 smithi093 ceph-mon[42070]: pgmap v166: 129 pgs: 129 active+clean; 608 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2023-12-14T18:31:56.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:55 smithi097 ceph-mon[43486]: pgmap v166: 129 pgs: 129 active+clean; 608 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 2.7 MiB/s wr, 269 op/s 2023-12-14T18:31:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:57 smithi160 ceph-mon[43840]: pgmap v167: 129 pgs: 129 active+clean; 629 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 506 op/s 2023-12-14T18:31:58.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:57 smithi093 ceph-mon[42070]: pgmap v167: 129 pgs: 129 active+clean; 629 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 506 op/s 2023-12-14T18:31:58.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:57 smithi097 ceph-mon[43486]: pgmap v167: 129 pgs: 129 active+clean; 629 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.5 MiB/s rd, 4.4 MiB/s wr, 506 op/s 2023-12-14T18:31:59.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:31:59 smithi160 ceph-mon[43840]: pgmap v168: 129 pgs: 129 active+clean; 629 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 348 op/s 2023-12-14T18:32:00.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:31:59 smithi093 ceph-mon[42070]: pgmap v168: 129 pgs: 129 active+clean; 629 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 348 op/s 2023-12-14T18:32:00.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:31:59 smithi097 ceph-mon[43486]: pgmap v168: 129 pgs: 129 active+clean; 629 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 2.9 MiB/s wr, 348 op/s 2023-12-14T18:32:01.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:01 smithi160 ceph-mon[43840]: pgmap v169: 129 pgs: 129 active+clean; 632 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 568 op/s 2023-12-14T18:32:02.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:01 smithi093 ceph-mon[42070]: pgmap v169: 129 pgs: 129 active+clean; 632 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 568 op/s 2023-12-14T18:32:02.097 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:01 smithi097 ceph-mon[43486]: pgmap v169: 129 pgs: 129 active+clean; 632 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.2 MiB/s wr, 568 op/s 2023-12-14T18:32:03.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:03 smithi160 ceph-mon[43840]: pgmap v170: 129 pgs: 129 active+clean; 632 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 457 op/s 2023-12-14T18:32:04.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:03 smithi093 ceph-mon[42070]: pgmap v170: 129 pgs: 129 active+clean; 632 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 457 op/s 2023-12-14T18:32:04.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:03 smithi097 ceph-mon[43486]: pgmap v170: 129 pgs: 129 active+clean; 632 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 457 op/s 2023-12-14T18:32:05.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:05 smithi160 ceph-mon[43840]: pgmap v171: 129 pgs: 129 active+clean; 632 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 457 op/s 2023-12-14T18:32:06.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:05 smithi093 ceph-mon[42070]: pgmap v171: 129 pgs: 129 active+clean; 632 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 457 op/s 2023-12-14T18:32:06.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:05 smithi097 ceph-mon[43486]: pgmap v171: 129 pgs: 129 active+clean; 632 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 MiB/s rd, 3.1 MiB/s wr, 457 op/s 2023-12-14T18:32:07.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:07 smithi160 ceph-mon[43840]: pgmap v172: 129 pgs: 129 active+clean; 652 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 609 op/s 2023-12-14T18:32:08.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:07 smithi093 ceph-mon[42070]: pgmap v172: 129 pgs: 129 active+clean; 652 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 609 op/s 2023-12-14T18:32:08.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:07 smithi097 ceph-mon[43486]: pgmap v172: 129 pgs: 129 active+clean; 652 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.8 MiB/s wr, 609 op/s 2023-12-14T18:32:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.787+0000 7f7f84fd4700 1 -- 172.21.15.93:0/3654956446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f80101c30 msgr2=0x7f7f8010a300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.792 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.787+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/3654956446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f80101c30 0x7f7f8010a300 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009990 tx=0x7f7f74009ca0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.788+0000 7f7f84fd4700 1 -- 172.21.15.93:0/3654956446 shutdown_connections 2023-12-14T18:32:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.788+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/3654956446 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f8010a840 0x7f7f8010cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.788+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/3654956446 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f80101320 0x7f7f801016f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.788+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/3654956446 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f80101c30 0x7f7f8010a300 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.788+0000 7f7f84fd4700 1 -- 172.21.15.93:0/3654956446 >> 172.21.15.93:0/3654956446 conn(0x7f7f80075fa0 msgr2=0x7f7f800763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.788+0000 7f7f84fd4700 1 -- 172.21.15.93:0/3654956446 shutdown_connections 2023-12-14T18:32:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.788+0000 7f7f84fd4700 1 -- 172.21.15.93:0/3654956446 wait complete. 2023-12-14T18:32:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.788+0000 7f7f84fd4700 1 Processor -- start 2023-12-14T18:32:08.793 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.788+0000 7f7f84fd4700 1 -- start start 2023-12-14T18:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f84fd4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80101320 0x7f7f801963d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f84fd4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f80101c30 0x7f7f80196910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f84fd4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f8010a840 0x7f7f8019ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f84fd4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019b3b0 con 0x7f7f8010a840 2023-12-14T18:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f84fd4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019b520 con 0x7f7f80101320 2023-12-14T18:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f84fd4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f8019b6e0 con 0x7f7f80101c30 2023-12-14T18:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f8010a840 0x7f7f8019ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80101320 0x7f7f801963d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.794 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f80101c30 0x7f7f80196910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80101320 0x7f7f801963d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:39520/0 (socket says 172.21.15.93:39520) 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7e59c700 1 -- 172.21.15.93:0/977130047 learned_addr learned my addr 172.21.15.93:0/977130047 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/977130047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80101320 msgr2=0x7f7f801963d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80101320 0x7f7f801963d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f8010a840 msgr2=0x7f7f8019ad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f8010a840 0x7f7f8019ad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/977130047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f740095f0 con 0x7f7f80101c30 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80101320 0x7f7f801963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7ed9d700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f8010a840 0x7f7f8019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.795 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f80101c30 0x7f7f80196910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009990 tx=0x7f7f74009580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.789+0000 7f7f6f7fe700 1 -- 172.21.15.93:0/977130047 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7401d070 con 0x7f7f80101c30 2023-12-14T18:32:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.790+0000 7f7f6f7fe700 1 -- 172.21.15.93:0/977130047 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f74021410 con 0x7f7f80101c30 2023-12-14T18:32:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.790+0000 7f7f84fd4700 1 -- 172.21.15.93:0/977130047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f80110310 con 0x7f7f80101c30 2023-12-14T18:32:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.790+0000 7f7f6f7fe700 1 -- 172.21.15.93:0/977130047 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f74017650 con 0x7f7f80101c30 2023-12-14T18:32:08.796 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.790+0000 7f7f84fd4700 1 -- 172.21.15.93:0/977130047 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f80110180 con 0x7f7f80101c30 2023-12-14T18:32:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.792+0000 7f7f6f7fe700 1 -- 172.21.15.93:0/977130047 <== mon.1 v2:172.21.15.160:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7f74021810 con 0x7f7f80101c30 2023-12-14T18:32:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.793+0000 7f7f6f7fe700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7f680a12c0 0x7f7f680a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.804 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.793+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7f680a12c0 0x7f7f680a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.793+0000 7f7f6f7fe700 1 -- 172.21.15.93:0/977130047 <== mon.1 v2:172.21.15.160:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f7f740ce310 con 0x7f7f80101c30 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.793+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7f680a12c0 0x7f7f680a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f70011550 tx=0x7f7f70009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.793+0000 7f7f91162b80 1 Processor -- start 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f91162b80 1 -- start start 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81893430 0x559e81a35e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 0x559e81a367e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a45500 0x559e81a478e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f91162b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81920580 con 0x559e81a45500 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f91162b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e8198f810 con 0x559e81893430 2023-12-14T18:32:08.805 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f91162b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81988ca0 con 0x559e81a36370 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a45500 0x559e81a478e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7e59c700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81893430 0x559e81a35e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7dd9b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 0x559e81a367e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7ed9d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a45500 0x559e81a478e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32860/0 (socket says 172.21.15.93:32860) 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7ed9d700 1 -- 172.21.15.93:0/2303124308 learned_addr learned my addr 172.21.15.93:0/2303124308 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/2303124308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81893430 msgr2=0x559e81a35e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/2303124308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81893430 0x559e81a35e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/2303124308 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a45500 msgr2=0x559e81a478e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/2303124308 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a45500 0x559e81a478e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/2303124308 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e8180a350 con 0x559e81a36370 2023-12-14T18:32:08.806 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.794+0000 7f7f7ed9d700 1 --2- 172.21.15.93:0/2303124308 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a45500 0x559e81a478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/2303124308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 0x559e81a367e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f740cb710 tx=0x7f7f74097530 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd1dad995a6c8a0c server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f6cff9700 1 -- 172.21.15.93:0/2303124308 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7409aa80 con 0x559e81a36370 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f6cff9700 1 -- 172.21.15.93:0/2303124308 <== mon.1 v2:172.21.15.160:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f7409b120 con 0x559e81a36370 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f6cff9700 1 -- 172.21.15.93:0/2303124308 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7409b5c0 con 0x559e81a36370 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f91162b80 1 -- 172.21.15.93:0/2303124308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 msgr2=0x559e81a367e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2303124308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 0x559e81a367e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f7f740cb710 tx=0x7f7f74097530 comp rx=0 tx=0).stop 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f91162b80 1 -- 172.21.15.93:0/2303124308 shutdown_connections 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2303124308 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a45500 0x559e81a478e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2303124308 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81893430 0x559e81a35e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2303124308 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 0x559e81a367e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.807 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.795+0000 7f7f91162b80 1 -- 172.21.15.93:0/2303124308 >> 172.21.15.93:0/2303124308 conn(0x559e81a2c890 msgr2=0x559e81a32200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 -- 172.21.15.93:0/2303124308 shutdown_connections 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 -- 172.21.15.93:0/2303124308 wait complete. 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 -- start start 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 Processor -- start 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a44e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a420d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a42610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81924f30 con 0x559e81a36370 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81924bd0 con 0x559e81a45500 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e8180c640 con 0x559e81893430 2023-12-14T18:32:08.808 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f91162b80 10 client.0 _ll_register_callbacks cb 0x559e819488b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f7dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a420d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f7dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a420d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32864/0 (socket says 172.21.15.93:32864) 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/2707124951 learned_addr learned my addr 172.21.15.93:0/2707124951 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f7ed9d700 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a42610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.796+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a44e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/2707124951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 msgr2=0x559e81a44e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a44e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/2707124951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 msgr2=0x559e81a42610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a42610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.809 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f7dd9b700 1 -- 172.21.15.93:0/2707124951 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f80110180 con 0x559e81a36370 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f7ed9d700 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a42610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a420d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7f740cb770 tx=0x7f7f740cb680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5d2a242b7eda601 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f65ffb700 1 -- 172.21.15.93:0/2707124951 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f7409b580 con 0x559e81a36370 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f65ffb700 1 -- 172.21.15.93:0/2707124951 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f7f7409f150 con 0x559e81a36370 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f91162b80 1 -- 172.21.15.93:0/2707124951 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559e81a29ea0 con 0x559e81a36370 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f65ffb700 1 -- 172.21.15.93:0/2707124951 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f7409aa80 con 0x559e81a36370 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f91162b80 10 client.15930 fetch_fsmap learned FSMap version 30 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f91162b80 1 -- 172.21.15.93:0/2707124951 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559e8194f110 con 0x559e81a36370 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f65ffb700 1 -- 172.21.15.93:0/2707124951 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x559e81a29ea0 con 0x559e81a36370 2023-12-14T18:32:08.810 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f65ffb700 1 -- 172.21.15.93:0/2707124951 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7f74097030 con 0x559e81a36370 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.797+0000 7f7f91162b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.798+0000 7f7f91162b80 10 client.15930 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.798+0000 7f7f91162b80 20 client.15930 populate_metadata read hostname 'smithi093' 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.798+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x559e81a54880 0x559e81a56c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.798+0000 7f7f91162b80 4 client.15930 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.798+0000 7f7f91162b80 1 -- 172.21.15.93:0/2707124951 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559e81981c80 con 0x559e81a54880 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.798+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x559e81a54880 0x559e81a56c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.798+0000 7f7f65ffb700 10 client.15930 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.798+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x559e81a54880 0x559e81a56c90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b94c350d917d15 server_cookie=64b1ad0406e783b in_seq=0 out_seq=0 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f65ffb700 1 -- 172.21.15.93:0/2707124951 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x559e81981c80 con 0x559e81a54880 2023-12-14T18:32:08.811 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f65ffb700 10 client.15930 handle_command_reply: tid=0 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 client.15930 shutdown 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 -- 172.21.15.93:0/2707124951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 msgr2=0x559e81a420d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a420d0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f7f740cb770 tx=0x7f7f740cb680 comp rx=0 tx=0).stop 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 -- 172.21.15.93:0/2707124951 shutdown_connections 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x559e81a54880 0x559e81a56c90 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a42610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f65ffb700 0 client.15930 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a44e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 --2- 172.21.15.93:0/2707124951 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a420d0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 -- 172.21.15.93:0/2707124951 >> 172.21.15.93:0/2707124951 conn(0x559e81a2c890 msgr2=0x559e81a31d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 -- 172.21.15.93:0/2707124951 shutdown_connections 2023-12-14T18:32:08.812 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 1 -- 172.21.15.93:0/2707124951 wait complete. 2023-12-14T18:32:08.813 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.799+0000 7f7f91162b80 20 client.15930 trim_cache size 0 max 16384 2023-12-14T18:32:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f91162b80 1 Processor -- start 2023-12-14T18:32:08.822 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f91162b80 1 -- start start 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81893430 0x559e81a4bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 0x559e81a448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a44e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f91162b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81a30ff0 con 0x559e81893430 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f91162b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81a478e0 con 0x559e81a45500 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f91162b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81a572f0 con 0x559e81a36370 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81893430 0x559e81a4bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7e59c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81893430 0x559e81a4bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32880/0 (socket says 172.21.15.93:32880) 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7e59c700 1 -- 172.21.15.93:0/3464716526 learned_addr learned my addr 172.21.15.93:0/3464716526 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7ed9d700 1 --2- 172.21.15.93:0/3464716526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a44e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.823 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/3464716526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 0x559e81a448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7e59c700 1 -- 172.21.15.93:0/3464716526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 msgr2=0x559e81a448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/3464716526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 0x559e81a448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7e59c700 1 -- 172.21.15.93:0/3464716526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 msgr2=0x559e81a44e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/3464716526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a44e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7e59c700 1 -- 172.21.15.93:0/3464716526 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559e8194ee60 con 0x559e81893430 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.815+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/3464716526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81893430 0x559e81a4bde0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x559e81a57560 tx=0x7f7f70014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28b6767aa35f9bf8 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f6d7fa700 1 -- 172.21.15.93:0/3464716526 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f70022030 con 0x559e81893430 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f6d7fa700 1 -- 172.21.15.93:0/3464716526 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f70023040 con 0x559e81893430 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f6d7fa700 1 -- 172.21.15.93:0/3464716526 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f70004e10 con 0x559e81893430 2023-12-14T18:32:08.824 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f91162b80 1 -- 172.21.15.93:0/3464716526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81893430 msgr2=0x559e81a4bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f91162b80 1 --2- 172.21.15.93:0/3464716526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81893430 0x559e81a4bde0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x559e81a57560 tx=0x7f7f70014000 comp rx=0 tx=0).stop 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f91162b80 1 -- 172.21.15.93:0/3464716526 shutdown_connections 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f91162b80 1 --2- 172.21.15.93:0/3464716526 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81893430 0x559e81a4bde0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f91162b80 1 --2- 172.21.15.93:0/3464716526 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a44e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f91162b80 1 --2- 172.21.15.93:0/3464716526 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81a36370 0x559e81a448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f91162b80 1 -- 172.21.15.93:0/3464716526 >> 172.21.15.93:0/3464716526 conn(0x559e81a2c890 msgr2=0x559e81a31d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f91162b80 1 -- 172.21.15.93:0/3464716526 shutdown_connections 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.816+0000 7f7f91162b80 1 -- 172.21.15.93:0/3464716526 wait complete. 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f91162b80 1 -- start start 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f91162b80 1 Processor -- start 2023-12-14T18:32:08.825 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a4b720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a4bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f91162b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a49bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f91162b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81a4a190 con 0x559e81a36370 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f91162b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81a4a300 con 0x559e81a45500 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f91162b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x559e81a4a4c0 con 0x559e81893430 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f91162b80 10 client.0 _ll_register_callbacks cb 0x7f7f78003a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f7dd9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a4bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f7e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a4b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f7e59c700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a4b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:53738/0 (socket says 172.21.15.93:53738) 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f7ed9d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a49bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.826 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f7e59c700 1 -- 172.21.15.93:0/1976253888 learned_addr learned my addr 172.21.15.93:0/1976253888 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a4bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:32896/0 (socket says 172.21.15.93:32896) 2023-12-14T18:32:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f7e59c700 1 -- 172.21.15.93:0/1976253888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 msgr2=0x559e81a49bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a49bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.817+0000 7f7f7e59c700 1 -- 172.21.15.93:0/1976253888 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 msgr2=0x559e81a4bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a4bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f7e59c700 1 -- 172.21.15.93:0/1976253888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f70017070 con 0x559e81893430 2023-12-14T18:32:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a4bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f7e59c700 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a4b720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7f70004780 tx=0x7f7f700047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2450d740dd84e2b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:08.827 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f65ffb700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f65ffb700 1 -- 172.21.15.93:0/1976253888 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f70022070 con 0x559e81893430 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f91162b80 1 -- 172.21.15.93:0/1976253888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f7f7409b120 con 0x559e81893430 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f65ffb700 1 -- 172.21.15.93:0/1976253888 <== mon.1 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f7f7000ecb0 con 0x559e81893430 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f65ffb700 1 -- 172.21.15.93:0/1976253888 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7f7002b300 con 0x559e81893430 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f91162b80 10 client.25555 fetch_fsmap learned FSMap version 30 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f91162b80 1 -- 172.21.15.93:0/1976253888 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x559e81935940 con 0x559e81893430 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f65ffb700 1 -- 172.21.15.93:0/1976253888 <== mon.1 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7f7409b120 con 0x559e81893430 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f65ffb700 1 -- 172.21.15.93:0/1976253888 <== mon.1 v2:172.21.15.160:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f7f7001d410 con 0x559e81893430 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f91162b80 10 client.25555 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.818+0000 7f7f91162b80 10 client.25555 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.819+0000 7f7f91162b80 20 client.25555 populate_metadata read hostname 'smithi093' 2023-12-14T18:32:08.828 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.819+0000 7f7f91162b80 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x559e81aa9800 0x559e81aabc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.819+0000 7f7f91162b80 4 client.25555 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:32:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.819+0000 7f7f91162b80 1 -- 172.21.15.93:0/1976253888 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x559e819819a0 con 0x559e81aa9800 2023-12-14T18:32:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.819+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x559e81aa9800 0x559e81aabc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.819+0000 7f7f65ffb700 10 client.25555 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:08.829 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:08.819+0000 7f7f7dd9b700 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x559e81aa9800 0x559e81aabc10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35911ddcbe3b7321 server_cookie=ab7aacb357d88379 in_seq=0 out_seq=0 2023-12-14T18:32:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.466+0000 7f7f65ffb700 1 -- 172.21.15.93:0/1976253888 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x559e819819a0 con 0x559e81aa9800 2023-12-14T18:32:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.466+0000 7f7f65ffb700 10 client.25555 handle_command_reply: tid=0 2023-12-14T18:32:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.466+0000 7f7f91162b80 1 client.25555 shutdown 2023-12-14T18:32:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.466+0000 7f7f91162b80 1 -- 172.21.15.93:0/1976253888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 msgr2=0x559e81a4b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.466+0000 7f7f91162b80 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a4b720 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f7f70004780 tx=0x7f7f700047b0 comp rx=0 tx=0).stop 2023-12-14T18:32:09.469 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.467+0000 7f7f91162b80 1 -- 172.21.15.93:0/1976253888 shutdown_connections 2023-12-14T18:32:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.467+0000 7f7f91162b80 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x559e81aa9800 0x559e81aabc10 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.467+0000 7f7f91162b80 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x559e81a45500 0x559e81a49bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.467+0000 7f7f91162b80 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x559e81893430 0x559e81a4b720 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.467+0000 7f7f91162b80 1 --2- 172.21.15.93:0/1976253888 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x559e81a36370 0x559e81a4bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.467+0000 7f7f91162b80 1 -- 172.21.15.93:0/1976253888 >> 172.21.15.93:0/1976253888 conn(0x559e81a2c890 msgr2=0x559e81a37f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.467+0000 7f7f65ffb700 0 client.25555 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.467+0000 7f7f91162b80 1 -- 172.21.15.93:0/1976253888 shutdown_connections 2023-12-14T18:32:09.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.467+0000 7f7f91162b80 1 -- 172.21.15.93:0/1976253888 wait complete. 2023-12-14T18:32:09.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.470+0000 7f7f91162b80 20 client.25555 trim_cache size 0 max 16384 2023-12-14T18:32:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:32:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:32:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:32:09.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:32:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.471+0000 7f7f84fd4700 1 -- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7f680a12c0 msgr2=0x7f7f680a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.471+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7f680a12c0 0x7f7f680a3770 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f7f70011550 tx=0x7f7f70009260 comp rx=0 tx=0).stop 2023-12-14T18:32:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.471+0000 7f7f84fd4700 1 -- 172.21.15.93:0/977130047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f80101c30 msgr2=0x7f7f80196910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:09.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.471+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f80101c30 0x7f7f80196910 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009990 tx=0x7f7f74009580 comp rx=0 tx=0).stop 2023-12-14T18:32:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.472+0000 7f7f84fd4700 1 -- 172.21.15.93:0/977130047 shutdown_connections 2023-12-14T18:32:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.472+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7f680a12c0 0x7f7f680a3770 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.472+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7f8010a840 0x7f7f8019ad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.472+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7f80101320 0x7f7f801963d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.472+0000 7f7f84fd4700 1 --2- 172.21.15.93:0/977130047 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7f80101c30 0x7f7f80196910 secure :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7f74009990 tx=0x7f7f74009580 comp rx=0 tx=0).stop 2023-12-14T18:32:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.472+0000 7f7f84fd4700 1 -- 172.21.15.93:0/977130047 >> 172.21.15.93:0/977130047 conn(0x7f7f80075fa0 msgr2=0x7f7f800fdbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:09.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.472+0000 7f7f84fd4700 1 -- 172.21.15.93:0/977130047 shutdown_connections 2023-12-14T18:32:09.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:09.472+0000 7f7f84fd4700 1 -- 172.21.15.93:0/977130047 wait complete. 2023-12-14T18:32:09.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:09 smithi160 ceph-mon[43840]: pgmap v173: 129 pgs: 129 active+clean; 652 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 371 op/s 2023-12-14T18:32:10.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:09 smithi093 ceph-mon[42070]: pgmap v173: 129 pgs: 129 active+clean; 652 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 371 op/s 2023-12-14T18:32:10.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:09 smithi097 ceph-mon[43486]: pgmap v173: 129 pgs: 129 active+clean; 652 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.0 MiB/s wr, 371 op/s 2023-12-14T18:32:11.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:11 smithi160 ceph-mon[43840]: pgmap v174: 129 pgs: 129 active+clean; 671 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 640 op/s 2023-12-14T18:32:12.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:11 smithi093 ceph-mon[42070]: pgmap v174: 129 pgs: 129 active+clean; 671 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 640 op/s 2023-12-14T18:32:12.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:11 smithi097 ceph-mon[43486]: pgmap v174: 129 pgs: 129 active+clean; 671 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.3 MiB/s rd, 4.7 MiB/s wr, 640 op/s 2023-12-14T18:32:13.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:13 smithi160 ceph-mon[43840]: pgmap v175: 129 pgs: 129 active+clean; 671 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 420 op/s 2023-12-14T18:32:14.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:13 smithi093 ceph-mon[42070]: pgmap v175: 129 pgs: 129 active+clean; 671 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 420 op/s 2023-12-14T18:32:14.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:13 smithi097 ceph-mon[43486]: pgmap v175: 129 pgs: 129 active+clean; 671 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 420 op/s 2023-12-14T18:32:15.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:15 smithi160 ceph-mon[43840]: pgmap v176: 129 pgs: 129 active+clean; 671 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 420 op/s 2023-12-14T18:32:16.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:15 smithi093 ceph-mon[42070]: pgmap v176: 129 pgs: 129 active+clean; 671 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 420 op/s 2023-12-14T18:32:16.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:15 smithi097 ceph-mon[43486]: pgmap v176: 129 pgs: 129 active+clean; 671 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 420 op/s 2023-12-14T18:32:18.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:17 smithi093 ceph-mon[42070]: pgmap v177: 129 pgs: 129 active+clean; 673 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 554 op/s 2023-12-14T18:32:18.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:17 smithi097 ceph-mon[43486]: pgmap v177: 129 pgs: 129 active+clean; 673 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 554 op/s 2023-12-14T18:32:18.248 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:17 smithi160 ceph-mon[43840]: pgmap v177: 129 pgs: 129 active+clean; 673 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 MiB/s rd, 4.5 MiB/s wr, 554 op/s 2023-12-14T18:32:20.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:19 smithi093 ceph-mon[42070]: pgmap v178: 129 pgs: 129 active+clean; 673 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 403 op/s 2023-12-14T18:32:20.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:19 smithi097 ceph-mon[43486]: pgmap v178: 129 pgs: 129 active+clean; 673 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 403 op/s 2023-12-14T18:32:20.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:19 smithi160 ceph-mon[43840]: pgmap v178: 129 pgs: 129 active+clean; 673 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 2.8 MiB/s wr, 403 op/s 2023-12-14T18:32:22.064 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:21 smithi093 ceph-mon[42070]: pgmap v179: 129 pgs: 129 active+clean; 686 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 550 op/s 2023-12-14T18:32:22.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:21 smithi097 ceph-mon[43486]: pgmap v179: 129 pgs: 129 active+clean; 686 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 550 op/s 2023-12-14T18:32:22.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:21 smithi160 ceph-mon[43840]: pgmap v179: 129 pgs: 129 active+clean; 686 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 MiB/s rd, 4.0 MiB/s wr, 550 op/s 2023-12-14T18:32:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.591+0000 7fbd53379700 1 -- 172.21.15.93:0/3778235408 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd4c0ff3a0 msgr2=0x7fbd4c0ff770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:22.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.591+0000 7fbd53379700 1 --2- 172.21.15.93:0/3778235408 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd4c0ff3a0 0x7fbd4c0ff770 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fbd480099c0 tx=0x7fbd48009cd0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.591+0000 7fbd53379700 1 -- 172.21.15.93:0/3778235408 shutdown_connections 2023-12-14T18:32:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.591+0000 7fbd53379700 1 --2- 172.21.15.93:0/3778235408 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd4c0ff3a0 0x7fbd4c0ff770 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.591+0000 7fbd53379700 1 --2- 172.21.15.93:0/3778235408 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd4c1043e0 0x7fbd4c10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.591+0000 7fbd53379700 1 --2- 172.21.15.93:0/3778235408 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd4c0ffd40 0x7fbd4c103d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.591+0000 7fbd53379700 1 -- 172.21.15.93:0/3778235408 >> 172.21.15.93:0/3778235408 conn(0x7fbd4c0fabf0 msgr2=0x7fbd4c0fd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.591+0000 7fbd53379700 1 -- 172.21.15.93:0/3778235408 shutdown_connections 2023-12-14T18:32:22.594 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.591+0000 7fbd53379700 1 -- 172.21.15.93:0/3778235408 wait complete. 2023-12-14T18:32:22.595 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd53379700 1 Processor -- start 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd53379700 1 -- start start 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd53379700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd4c0ff3a0 0x7fbd4c195290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd53379700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd4c0ffd40 0x7fbd4c1957d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd53379700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd4c1043e0 0x7fbd4c199c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd53379700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c19a270 con 0x7fbd4c0ff3a0 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd53379700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c19a3e0 con 0x7fbd4c0ffd40 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd53379700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c19a5a0 con 0x7fbd4c1043e0 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd51115700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd4c0ff3a0 0x7fbd4c195290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd50914700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd4c0ffd40 0x7fbd4c1957d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd51115700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd4c0ff3a0 0x7fbd4c195290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50852/0 (socket says 172.21.15.93:50852) 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.592+0000 7fbd51115700 1 -- 172.21.15.93:0/1124651912 learned_addr learned my addr 172.21.15.93:0/1124651912 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd51916700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd4c1043e0 0x7fbd4c199c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd50914700 1 -- 172.21.15.93:0/1124651912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd4c1043e0 msgr2=0x7fbd4c199c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:22.596 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd50914700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd4c1043e0 0x7fbd4c199c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd50914700 1 -- 172.21.15.93:0/1124651912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd4c0ff3a0 msgr2=0x7fbd4c195290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd50914700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd4c0ff3a0 0x7fbd4c195290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd50914700 1 -- 172.21.15.93:0/1124651912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd480095f0 con 0x7fbd4c0ffd40 2023-12-14T18:32:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd50914700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd4c0ffd40 0x7fbd4c1957d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c100110 tx=0x7fbd3c00eca0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd427fc700 1 -- 172.21.15.93:0/1124651912 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c010900 con 0x7fbd4c0ffd40 2023-12-14T18:32:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd53379700 1 -- 172.21.15.93:0/1124651912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4c10f1a0 con 0x7fbd4c0ffd40 2023-12-14T18:32:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd427fc700 1 -- 172.21.15.93:0/1124651912 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd3c00c430 con 0x7fbd4c0ffd40 2023-12-14T18:32:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd427fc700 1 -- 172.21.15.93:0/1124651912 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c018680 con 0x7fbd4c0ffd40 2023-12-14T18:32:22.597 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.593+0000 7fbd53379700 1 -- 172.21.15.93:0/1124651912 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c10f010 con 0x7fbd4c0ffd40 2023-12-14T18:32:22.598 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.596+0000 7fbd427fc700 1 -- 172.21.15.93:0/1124651912 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbd3c00c5a0 con 0x7fbd4c0ffd40 2023-12-14T18:32:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.597+0000 7fbd427fc700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd380a12c0 0x7fbd380a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.597+0000 7fbd51115700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd380a12c0 0x7fbd380a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.599 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.597+0000 7fbd427fc700 1 -- 172.21.15.93:0/1124651912 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fbd3c014070 con 0x7fbd4c0ffd40 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd5f507b80 1 Processor -- start 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd5f507b80 1 -- start start 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51115700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd380a12c0 0x7fbd380a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd480036c0 tx=0x7fbd48005fb0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b369e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b36a370 0x557f2b36a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 0x557f2b37b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b254580 con 0x557f2b1c7430 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b2c3810 con 0x557f2b36a370 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b2bcca0 con 0x557f2b379500 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51115700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b369e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51115700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b369e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50866/0 (socket says 172.21.15.93:50866) 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51115700 1 -- 172.21.15.93:0/3719478811 learned_addr learned my addr 172.21.15.93:0/3719478811 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:22.602 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd50914700 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b36a370 0x557f2b36a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51916700 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 0x557f2b37b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51115700 1 -- 172.21.15.93:0/3719478811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 msgr2=0x557f2b37b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51115700 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 0x557f2b37b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51115700 1 -- 172.21.15.93:0/3719478811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b36a370 msgr2=0x557f2b36a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51115700 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b36a370 0x557f2b36a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd51115700 1 -- 172.21.15.93:0/3719478811 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f2b13e350 con 0x557f2b1c7430 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.598+0000 7fbd50914700 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b36a370 0x557f2b36a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.599+0000 7fbd51115700 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b369e30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbd48003b50 tx=0x7fbd4801f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2834084abf493192 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.599+0000 7fbd377fe700 1 -- 172.21.15.93:0/3719478811 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4800ba10 con 0x557f2b1c7430 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.599+0000 7fbd377fe700 1 -- 172.21.15.93:0/3719478811 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd4801e040 con 0x557f2b1c7430 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.599+0000 7fbd377fe700 1 -- 172.21.15.93:0/3719478811 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd480306e0 con 0x557f2b1c7430 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.599+0000 7fbd5f507b80 1 -- 172.21.15.93:0/3719478811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 msgr2=0x557f2b369e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.599+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b369e30 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fbd48003b50 tx=0x7fbd4801f000 comp rx=0 tx=0).stop 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 -- 172.21.15.93:0/3719478811 shutdown_connections 2023-12-14T18:32:22.603 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b369e30 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 0x557f2b37b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/3719478811 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b36a370 0x557f2b36a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 -- 172.21.15.93:0/3719478811 >> 172.21.15.93:0/3719478811 conn(0x557f2b360890 msgr2=0x557f2b366200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 -- 172.21.15.93:0/3719478811 shutdown_connections 2023-12-14T18:32:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 -- 172.21.15.93:0/3719478811 wait complete. 2023-12-14T18:32:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 -- start start 2023-12-14T18:32:22.604 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 Processor -- start 2023-12-14T18:32:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b378e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 0x557f2b376140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 0x557f2b3766e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.607 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b258f30 con 0x557f2b1c7430 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b258bd0 con 0x557f2b379500 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b140640 con 0x557f2b36a370 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.600+0000 7fbd5f507b80 10 client.0 _ll_register_callbacks cb 0x557f2b27c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51115700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b378e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51115700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b378e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50868/0 (socket says 172.21.15.93:50868) 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51115700 1 -- 172.21.15.93:0/726733219 learned_addr learned my addr 172.21.15.93:0/726733219 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd50914700 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 0x557f2b376140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51916700 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 0x557f2b3766e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51115700 1 -- 172.21.15.93:0/726733219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 msgr2=0x557f2b376140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51115700 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 0x557f2b376140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51115700 1 -- 172.21.15.93:0/726733219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 msgr2=0x557f2b3766e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51115700 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 0x557f2b3766e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51115700 1 -- 172.21.15.93:0/726733219 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c10c7f0 con 0x557f2b1c7430 2023-12-14T18:32:22.608 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd51115700 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b378e20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800b600 tx=0x7fbd48005a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74494ef12097e430 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd34ff9700 1 -- 172.21.15.93:0/726733219 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48030ba0 con 0x557f2b1c7430 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd34ff9700 1 -- 172.21.15.93:0/726733219 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fbd48030d00 con 0x557f2b1c7430 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd5f507b80 1 -- 172.21.15.93:0/726733219 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557f2b35dea0 con 0x557f2b1c7430 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd34ff9700 1 -- 172.21.15.93:0/726733219 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd48030790 con 0x557f2b1c7430 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd5f507b80 10 client.15948 fetch_fsmap learned FSMap version 30 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd5f507b80 1 -- 172.21.15.93:0/726733219 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557f2b283110 con 0x557f2b1c7430 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.601+0000 7fbd34ff9700 1 -- 172.21.15.93:0/726733219 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x557f2b35dea0 con 0x557f2b1c7430 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd34ff9700 1 -- 172.21.15.93:0/726733219 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fbd4800f980 con 0x557f2b1c7430 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd5f507b80 10 client.15948 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd5f507b80 10 client.15948 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd5f507b80 20 client.15948 populate_metadata read hostname 'smithi093' 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557f2b388a70 0x557f2b38ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.609 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd5f507b80 4 client.15948 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd5f507b80 1 -- 172.21.15.93:0/726733219 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557f2b2b5c80 con 0x557f2b388a70 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd50914700 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557f2b388a70 0x557f2b38ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd50914700 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557f2b388a70 0x557f2b38ae80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96192f784694c7d7 server_cookie=cfc6440204201431 in_seq=0 out_seq=0 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.602+0000 7fbd34ff9700 10 client.15948 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd34ff9700 1 -- 172.21.15.93:0/726733219 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x557f2b2b5c80 con 0x557f2b388a70 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd34ff9700 10 client.15948 handle_command_reply: tid=0 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 client.15948 shutdown 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 -- 172.21.15.93:0/726733219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 msgr2=0x557f2b378e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b378e20 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fbd4800b600 tx=0x7fbd48005a00 comp rx=0 tx=0).stop 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 -- 172.21.15.93:0/726733219 shutdown_connections 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557f2b388a70 0x557f2b38ae80 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b378e20 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 0x557f2b3766e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd34ff9700 0 client.15948 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:22.610 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/726733219 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 0x557f2b376140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 -- 172.21.15.93:0/726733219 >> 172.21.15.93:0/726733219 conn(0x557f2b360890 msgr2=0x557f2b365d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 -- 172.21.15.93:0/726733219 shutdown_connections 2023-12-14T18:32:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 1 -- 172.21.15.93:0/726733219 wait complete. 2023-12-14T18:32:22.611 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.603+0000 7fbd5f507b80 20 client.15948 trim_cache size 0 max 16384 2023-12-14T18:32:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.618+0000 7fbd5f507b80 1 Processor -- start 2023-12-14T18:32:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.618+0000 7fbd5f507b80 1 -- start start 2023-12-14T18:32:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.618+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b1c7430 0x557f2b380040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b36a370 0x557f2b385150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 0x557f2b3714a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.621 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b36b8c0 con 0x557f2b36a370 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b36bd00 con 0x557f2b1c7430 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b36a740 con 0x557f2b379500 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd50914700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b36a370 0x557f2b385150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd50914700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b36a370 0x557f2b385150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50884/0 (socket says 172.21.15.93:50884) 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd50914700 1 -- 172.21.15.93:0/2726143430 learned_addr learned my addr 172.21.15.93:0/2726143430 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd51916700 1 --2- 172.21.15.93:0/2726143430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 0x557f2b3714a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd51115700 1 --2- 172.21.15.93:0/2726143430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b1c7430 0x557f2b380040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd50914700 1 -- 172.21.15.93:0/2726143430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 msgr2=0x557f2b3714a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd50914700 1 --2- 172.21.15.93:0/2726143430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 0x557f2b3714a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd50914700 1 -- 172.21.15.93:0/2726143430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b1c7430 msgr2=0x557f2b380040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd50914700 1 --2- 172.21.15.93:0/2726143430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b1c7430 0x557f2b380040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd50914700 1 -- 172.21.15.93:0/2726143430 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557f2b36ac30 con 0x557f2b36a370 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd50914700 1 --2- 172.21.15.93:0/2726143430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b36a370 0x557f2b385150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c099e40 tx=0x7fbd3c01a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e23cfce49c22b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd37fff700 1 -- 172.21.15.93:0/2726143430 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c0b6070 con 0x557f2b36a370 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd37fff700 1 -- 172.21.15.93:0/2726143430 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd3c099eb0 con 0x557f2b36a370 2023-12-14T18:32:22.622 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.619+0000 7fbd37fff700 1 -- 172.21.15.93:0/2726143430 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd3c09a110 con 0x557f2b36a370 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 -- 172.21.15.93:0/2726143430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b36a370 msgr2=0x557f2b385150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/2726143430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b36a370 0x557f2b385150 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fbd3c099e40 tx=0x7fbd3c01a040 comp rx=0 tx=0).stop 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 -- 172.21.15.93:0/2726143430 shutdown_connections 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/2726143430 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b379500 0x557f2b3714a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/2726143430 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b1c7430 0x557f2b380040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/2726143430 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b36a370 0x557f2b385150 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 -- 172.21.15.93:0/2726143430 >> 172.21.15.93:0/2726143430 conn(0x557f2b360890 msgr2=0x557f2b364930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 -- 172.21.15.93:0/2726143430 shutdown_connections 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 -- 172.21.15.93:0/2726143430 wait complete. 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.620+0000 7fbd5f507b80 1 -- start start 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd5f507b80 1 Processor -- start 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b37e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 0x557f2b37e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.626 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd5f507b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 0x557f2b3db8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b37eee0 con 0x557f2b1c7430 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b3dbe60 con 0x557f2b379500 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd5f507b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x557f2b3dc020 con 0x557f2b36a370 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd5f507b80 10 client.0 _ll_register_callbacks cb 0x7fbd3c095ad0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd51115700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b37e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd50914700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 0x557f2b37e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd51115700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b37e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50900/0 (socket says 172.21.15.93:50900) 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd51115700 1 -- 172.21.15.93:0/279163669 learned_addr learned my addr 172.21.15.93:0/279163669 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.621+0000 7fbd51916700 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 0x557f2b3db8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd51115700 1 -- 172.21.15.93:0/279163669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 msgr2=0x557f2b37e750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd51115700 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 0x557f2b37e750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd51115700 1 -- 172.21.15.93:0/279163669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 msgr2=0x557f2b3db8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd51115700 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 0x557f2b3db8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd51115700 1 -- 172.21.15.93:0/279163669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c10f1a0 con 0x557f2b1c7430 2023-12-14T18:32:22.627 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd50914700 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 0x557f2b37e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd51115700 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b37e210 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fbd48004750 tx=0x7fbd4800fb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=636d0c050aadb6ad server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd34ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd34ff9700 1 -- 172.21.15.93:0/279163669 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd48030ba0 con 0x557f2b1c7430 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd5f507b80 1 -- 172.21.15.93:0/279163669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbd4801e040 con 0x557f2b1c7430 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd34ff9700 1 -- 172.21.15.93:0/279163669 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fbd48030d00 con 0x557f2b1c7430 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd34ff9700 1 -- 172.21.15.93:0/279163669 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbd4800b790 con 0x557f2b1c7430 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd5f507b80 10 client.15954 fetch_fsmap learned FSMap version 30 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd5f507b80 1 -- 172.21.15.93:0/279163669 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x557f2b0eab00 con 0x557f2b1c7430 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd34ff9700 1 -- 172.21.15.93:0/279163669 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd4801e040 con 0x557f2b1c7430 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd34ff9700 1 -- 172.21.15.93:0/279163669 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fbd480266e0 con 0x557f2b1c7430 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd5f507b80 10 client.15954 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd5f507b80 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd5f507b80 20 client.15954 populate_metadata read hostname 'smithi093' 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557f2b3dd470 0x557f2b3df880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:22.628 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd5f507b80 4 client.15954 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:32:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.622+0000 7fbd5f507b80 1 -- 172.21.15.93:0/279163669 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x557f2b2b59a0 con 0x557f2b3dd470 2023-12-14T18:32:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.623+0000 7fbd50914700 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557f2b3dd470 0x557f2b3df880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.623+0000 7fbd50914700 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557f2b3dd470 0x557f2b3df880 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83847fc9d1b35f31 server_cookie=f2c436f614db9043 in_seq=0 out_seq=0 2023-12-14T18:32:22.629 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:22.623+0000 7fbd34ff9700 10 client.15954 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:23.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:22 smithi097 ceph-mon[43486]: pgmap v180: 129 pgs: 129 active+clean; 686 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2023-12-14T18:32:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:22 smithi160 ceph-mon[43840]: pgmap v180: 129 pgs: 129 active+clean; 686 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2023-12-14T18:32:23.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:22 smithi093 ceph-mon[42070]: pgmap v180: 129 pgs: 129 active+clean; 686 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd34ff9700 1 -- 172.21.15.93:0/279163669 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x557f2b2b59a0 con 0x557f2b3dd470 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd34ff9700 10 client.15954 handle_command_reply: tid=0 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd5f507b80 1 client.15954 shutdown 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd5f507b80 1 -- 172.21.15.93:0/279163669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 msgr2=0x557f2b37e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b37e210 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7fbd48004750 tx=0x7fbd4800fb40 comp rx=0 tx=0).stop 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd5f507b80 1 -- 172.21.15.93:0/279163669 shutdown_connections 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x557f2b3dd470 0x557f2b3df880 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x557f2b1c7430 0x557f2b37e210 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x557f2b379500 0x557f2b3db8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd5f507b80 1 --2- 172.21.15.93:0/279163669 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x557f2b36a370 0x557f2b37e750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd34ff9700 0 client.15954 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.776+0000 7fbd5f507b80 1 -- 172.21.15.93:0/279163669 >> 172.21.15.93:0/279163669 conn(0x557f2b360890 msgr2=0x557f2b3714a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd5f507b80 1 -- 172.21.15.93:0/279163669 shutdown_connections 2023-12-14T18:32:23.779 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd5f507b80 1 -- 172.21.15.93:0/279163669 wait complete. 2023-12-14T18:32:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd5f507b80 20 client.15954 trim_cache size 0 max 16384 2023-12-14T18:32:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:32:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:32:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:32:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:32:23.780 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 -- 172.21.15.93:0/1124651912 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd380a12c0 msgr2=0x7fbd380a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd380a12c0 0x7fbd380a3770 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fbd480036c0 tx=0x7fbd48005fb0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 -- 172.21.15.93:0/1124651912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd4c0ffd40 msgr2=0x7fbd4c1957d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd4c0ffd40 0x7fbd4c1957d0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c100110 tx=0x7fbd3c00eca0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 -- 172.21.15.93:0/1124651912 shutdown_connections 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd380a12c0 0x7fbd380a3770 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd4c0ff3a0 0x7fbd4c195290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd4c1043e0 0x7fbd4c199c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 --2- 172.21.15.93:0/1124651912 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd4c0ffd40 0x7fbd4c1957d0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 -- 172.21.15.93:0/1124651912 >> 172.21.15.93:0/1124651912 conn(0x7fbd4c0fabf0 msgr2=0x7fbd4c102d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 -- 172.21.15.93:0/1124651912 shutdown_connections 2023-12-14T18:32:23.781 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:23.777+0000 7fbd53379700 1 -- 172.21.15.93:0/1124651912 wait complete. 2023-12-14T18:32:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:24 smithi160 ceph-mon[43840]: pgmap v181: 129 pgs: 129 active+clean; 686 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2023-12-14T18:32:25.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:24 smithi093 ceph-mon[42070]: pgmap v181: 129 pgs: 129 active+clean; 686 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2023-12-14T18:32:25.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:24 smithi097 ceph-mon[43486]: pgmap v181: 129 pgs: 129 active+clean; 686 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 2.3 MiB/s wr, 281 op/s 2023-12-14T18:32:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:26 smithi160 ceph-mon[43840]: pgmap v182: 129 pgs: 129 active+clean; 696 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 455 op/s 2023-12-14T18:32:27.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:26 smithi093 ceph-mon[42070]: pgmap v182: 129 pgs: 129 active+clean; 696 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 455 op/s 2023-12-14T18:32:27.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:26 smithi097 ceph-mon[43486]: pgmap v182: 129 pgs: 129 active+clean; 696 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 3.5 MiB/s wr, 455 op/s 2023-12-14T18:32:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:28 smithi160 ceph-mon[43840]: pgmap v183: 129 pgs: 129 active+clean; 696 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 321 op/s 2023-12-14T18:32:29.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:28 smithi093 ceph-mon[42070]: pgmap v183: 129 pgs: 129 active+clean; 696 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 321 op/s 2023-12-14T18:32:29.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:28 smithi097 ceph-mon[43486]: pgmap v183: 129 pgs: 129 active+clean; 696 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.3 MiB/s wr, 321 op/s 2023-12-14T18:32:31.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:30 smithi160 ceph-mon[43840]: pgmap v184: 129 pgs: 129 active+clean; 707 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 370 op/s 2023-12-14T18:32:31.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:30 smithi093 ceph-mon[42070]: pgmap v184: 129 pgs: 129 active+clean; 707 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 370 op/s 2023-12-14T18:32:31.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:30 smithi097 ceph-mon[43486]: pgmap v184: 129 pgs: 129 active+clean; 707 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.3 MiB/s rd, 3.2 MiB/s wr, 370 op/s 2023-12-14T18:32:33.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:32 smithi160 ceph-mon[43840]: pgmap v185: 129 pgs: 129 active+clean; 707 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 222 op/s 2023-12-14T18:32:33.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:32 smithi093 ceph-mon[42070]: pgmap v185: 129 pgs: 129 active+clean; 707 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 222 op/s 2023-12-14T18:32:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:32 smithi097 ceph-mon[43486]: pgmap v185: 129 pgs: 129 active+clean; 707 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 222 op/s 2023-12-14T18:32:35.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:34 smithi160 ceph-mon[43840]: pgmap v186: 129 pgs: 129 active+clean; 707 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 222 op/s 2023-12-14T18:32:35.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:34 smithi093 ceph-mon[42070]: pgmap v186: 129 pgs: 129 active+clean; 707 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 222 op/s 2023-12-14T18:32:35.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:34 smithi097 ceph-mon[43486]: pgmap v186: 129 pgs: 129 active+clean; 707 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.1 MiB/s wr, 222 op/s 2023-12-14T18:32:36.130 INFO:tasks.workunit.client.0.smithi093.stdout:cleaned up hierarchy 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.889+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3493345512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 msgr2=0x7fb4dc10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.889+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3493345512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 0x7fb4dc10bfd0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7fb4d400b0d0 tx=0x7fb4d400b3e0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.889+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3493345512 shutdown_connections 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.889+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3493345512 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 0x7fb4dc10bfd0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.889+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3493345512 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4dc0ff3c0 0x7fb4dc0ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.889+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3493345512 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4dc0ffd60 0x7fb4dc103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.889+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3493345512 >> 172.21.15.93:0/3493345512 conn(0x7fb4dc0763a0 msgr2=0x7fb4dc0767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.889+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3493345512 shutdown_connections 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.889+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3493345512 wait complete. 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e3a47700 1 Processor -- start 2023-12-14T18:32:36.892 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e3a47700 1 -- start start 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e3a47700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4dc0ff3c0 0x7fb4dc1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e3a47700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4dc0ffd60 0x7fb4dc198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e3a47700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 0x7fb4dc19bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e3a47700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc19c5d0 con 0x7fb4dc104400 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e3a47700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc19c740 con 0x7fb4dc0ffd60 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e3a47700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4dc19c900 con 0x7fb4dc0ff3c0 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 0x7fb4dc19bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 0x7fb4dc19bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:51424/0 (socket says 172.21.15.93:51424) 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e17e3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4dc0ff3c0 0x7fb4dc1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/3387183213 learned_addr learned my addr 172.21.15.93:0/3387183213 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e0fe2700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4dc0ffd60 0x7fb4dc198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/3387183213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4dc0ff3c0 msgr2=0x7fb4dc1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4dc0ff3c0 0x7fb4dc1986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/3387183213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4dc0ffd60 msgr2=0x7fb4dc198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:36.893 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4dc0ffd60 0x7fb4dc198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/3387183213 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4d4009d00 con 0x7fb4dc104400 2023-12-14T18:32:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 0x7fb4dc19bfd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4007870 tx=0x7fb4d40078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.890+0000 7fb4d27fc700 1 -- 172.21.15.93:0/3387183213 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d400e040 con 0x7fb4dc104400 2023-12-14T18:32:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.891+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3387183213 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4dc10f1d0 con 0x7fb4dc104400 2023-12-14T18:32:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.891+0000 7fb4d27fc700 1 -- 172.21.15.93:0/3387183213 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d4005070 con 0x7fb4dc104400 2023-12-14T18:32:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.891+0000 7fb4d27fc700 1 -- 172.21.15.93:0/3387183213 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d401b6c0 con 0x7fb4dc104400 2023-12-14T18:32:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.891+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3387183213 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc10f040 con 0x7fb4dc104400 2023-12-14T18:32:36.894 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.893+0000 7fb4d27fc700 1 -- 172.21.15.93:0/3387183213 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fb4d40051e0 con 0x7fb4dc104400 2023-12-14T18:32:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.894+0000 7fb4d27fc700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb4c80a12c0 0x7fb4c80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.894+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb4c80a12c0 0x7fb4c80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.894+0000 7fb4d27fc700 1 -- 172.21.15.93:0/3387183213 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb4d40ce130 con 0x7fb4dc104400 2023-12-14T18:32:36.895 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.894+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb4c80a12c0 0x7fb4c80a3770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009c60 tx=0x7fb4cc009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4efbd5b80 1 Processor -- start 2023-12-14T18:32:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4efbd5b80 1 -- start start 2023-12-14T18:32:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 0x55c6dc4b6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 0x55c6dc4b77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc3a1580 con 0x55c6dc4c20f0 2023-12-14T18:32:36.897 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc410810 con 0x55c6dc4b7370 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc409ca0 con 0x55c6dc314430 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4e17e3700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 0x55c6dc4b6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c44d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:51428/0 (socket says 172.21.15.93:51428) 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4e0fe2700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 0x55c6dc4b77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/2190795656 learned_addr learned my addr 172.21.15.93:0/2190795656 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/2190795656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 msgr2=0x55c6dc4b6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/2190795656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 0x55c6dc4b6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/2190795656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 msgr2=0x55c6dc4b77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.895+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/2190795656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 0x55c6dc4b77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.896+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/2190795656 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6dc28b350 con 0x55c6dc4c20f0 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.896+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/2190795656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c44d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4097a10 tx=0x7fb4d409aae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57dc0e3380e1d61b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.896+0000 7fb4c77fe700 1 -- 172.21.15.93:0/2190795656 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d409efd0 con 0x55c6dc4c20f0 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.896+0000 7fb4c77fe700 1 -- 172.21.15.93:0/2190795656 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d40a3360 con 0x55c6dc4c20f0 2023-12-14T18:32:36.898 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.896+0000 7fb4c77fe700 1 -- 172.21.15.93:0/2190795656 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d409a490 con 0x55c6dc4c20f0 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.896+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/2190795656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 msgr2=0x55c6dc4c44d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.896+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/2190795656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c44d0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4097a10 tx=0x7fb4d409aae0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/2190795656 shutdown_connections 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/2190795656 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c44d0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/2190795656 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 0x55c6dc4b6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/2190795656 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 0x55c6dc4b77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/2190795656 >> 172.21.15.93:0/2190795656 conn(0x55c6dc4ad890 msgr2=0x55c6dc4b3200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/2190795656 shutdown_connections 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/2190795656 wait complete. 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 -- start start 2023-12-14T18:32:36.902 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 Processor -- start 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4c1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 0x55c6dc4bee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4bf4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc3a5f30 con 0x55c6dc4c20f0 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc3a5bd0 con 0x55c6dc314430 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc28d640 con 0x55c6dc4b7370 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.897+0000 7fb4efbd5b80 10 client.0 _ll_register_callbacks cb 0x55c6dc3c98b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e17e3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4c1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4bf4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e17e3700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4c1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:44976/0 (socket says 172.21.15.93:44976) 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4bf4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:51436/0 (socket says 172.21.15.93:51436) 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e17e3700 1 -- 172.21.15.93:0/4207577518 learned_addr learned my addr 172.21.15.93:0/4207577518 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e0fe2700 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 0x55c6dc4bee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e17e3700 1 -- 172.21.15.93:0/4207577518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 msgr2=0x55c6dc4bee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 0x55c6dc4bee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.903 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e17e3700 1 -- 172.21.15.93:0/4207577518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 msgr2=0x55c6dc4bf4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4bf4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e17e3700 1 -- 172.21.15.93:0/4207577518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4dc10f040 con 0x55c6dc314430 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4bf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4c1ab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc005910 tx=0x7fb4cc0047c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4a72cc4843b937c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.97:3300/0 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/4207577518 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4cc022070 con 0x55c6dc314430 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/4207577518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c6dc4aaea0 con 0x55c6dc314430 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/4207577518 <== mon.2 v2:172.21.15.97:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb4cc01d620 con 0x55c6dc314430 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/4207577518 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4cc01d7e0 con 0x55c6dc314430 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4efbd5b80 10 client.25583 fetch_fsmap learned FSMap version 30 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.898+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/4207577518 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c6dc3d0110 con 0x55c6dc314430 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/4207577518 <== mon.2 v2:172.21.15.97:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55c6dc4aaea0 con 0x55c6dc314430 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/4207577518 <== mon.2 v2:172.21.15.97:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fb4cc027a00 con 0x55c6dc314430 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4efbd5b80 10 client.25583 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4efbd5b80 10 client.25583 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:32:36.904 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4efbd5b80 20 client.25583 populate_metadata read hostname 'smithi093' 2023-12-14T18:32:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c6dc4d1aa0 0x55c6dc4d3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4efbd5b80 4 client.25583 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/4207577518 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c6dc402c80 con 0x55c6dc4d1aa0 2023-12-14T18:32:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4e0fe2700 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c6dc4d1aa0 0x55c6dc4d3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4c4ff9700 10 client.25583 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:36.905 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.899+0000 7fb4e0fe2700 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c6dc4d1aa0 0x55c6dc4d3eb0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa9684d055b5019 server_cookie=727b9ac3d7eec655 in_seq=0 out_seq=0 2023-12-14T18:32:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/4207577518 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55c6dc402c80 con 0x55c6dc4d1aa0 2023-12-14T18:32:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4c4ff9700 10 client.25583 handle_command_reply: tid=0 2023-12-14T18:32:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 client.25583 shutdown 2023-12-14T18:32:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/4207577518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 msgr2=0x55c6dc4c1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4c1ab0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc005910 tx=0x7fb4cc0047c0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/4207577518 shutdown_connections 2023-12-14T18:32:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c6dc4d1aa0 0x55c6dc4d3eb0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4bf4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.906 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4c4ff9700 0 client.25583 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4c1ab0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/4207577518 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 0x55c6dc4bee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/4207577518 >> 172.21.15.93:0/4207577518 conn(0x55c6dc4ad890 msgr2=0x55c6dc4b2d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/4207577518 shutdown_connections 2023-12-14T18:32:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/4207577518 wait complete. 2023-12-14T18:32:36.907 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.902+0000 7fb4efbd5b80 20 client.25583 trim_cache size 0 max 16384 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4efbd5b80 1 Processor -- start 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4efbd5b80 1 -- start start 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4d3e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 0x55c6dc4cdbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c15e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc4ad4d0 con 0x55c6dc4c20f0 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc4bacb0 con 0x55c6dc314430 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc4bfac0 con 0x55c6dc4b7370 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c15e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.922 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c15e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:51452/0 (socket says 172.21.15.93:51452) 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/127309414 learned_addr learned my addr 172.21.15.93:0/127309414 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e0fe2700 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 0x55c6dc4cdbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4d3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/127309414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 msgr2=0x55c6dc4cdbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 0x55c6dc4cdbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/127309414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 msgr2=0x55c6dc4d3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4d3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/127309414 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c6dc3cfe60 con 0x55c6dc4c20f0 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4d3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c15e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb4d409af30 tx=0x7fb4d40099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56aad03df5f14b6b server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.917+0000 7fb4c7fff700 1 -- 172.21.15.93:0/127309414 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d409efd0 con 0x55c6dc4c20f0 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4c7fff700 1 -- 172.21.15.93:0/127309414 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d40054c0 con 0x55c6dc4c20f0 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4c7fff700 1 -- 172.21.15.93:0/127309414 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d409b200 con 0x55c6dc4c20f0 2023-12-14T18:32:36.923 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/127309414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 msgr2=0x55c6dc4c15e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c15e0 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7fb4d409af30 tx=0x7fb4d40099e0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/127309414 shutdown_connections 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c15e0 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc314430 0x55c6dc4d3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/127309414 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc4b7370 0x55c6dc4cdbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/127309414 >> 172.21.15.93:0/127309414 conn(0x55c6dc4ad890 msgr2=0x55c6dc4d87d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/127309414 shutdown_connections 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/127309414 wait complete. 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.918+0000 7fb4efbd5b80 1 -- start start 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4efbd5b80 1 Processor -- start 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 0x55c6dc4c62a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 0x55c6dc4c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4efbd5b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc5249e0 con 0x55c6dc4c20f0 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc524b50 con 0x55c6dc4b7370 2023-12-14T18:32:36.924 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4efbd5b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c6dc524d10 con 0x55c6dc314430 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4efbd5b80 10 client.0 _ll_register_callbacks cb 0x7fb4cc0044e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e1fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:51458/0 (socket says 172.21.15.93:51458) 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/3428970294 learned_addr learned my addr 172.21.15.93:0/3428970294 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 0x55c6dc4c62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e0fe2700 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 0x55c6dc4c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/3428970294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 msgr2=0x55c6dc4c62a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 0x55c6dc4c62a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/3428970294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 msgr2=0x55c6dc4c67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 0x55c6dc4c67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e1fe4700 1 -- 172.21.15.93:0/3428970294 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c6dc28b350 con 0x55c6dc4c20f0 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4e1fe4700 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c6d20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40974a0 tx=0x7fb4d40b93f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df4333ab6ce12793 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:36.925 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.919+0000 7fb4c4ff9700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/3428970294 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4d4009b80 con 0x55c6dc4c20f0 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/3428970294 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fb4d409efd0 con 0x55c6dc4c20f0 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/3428970294 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fb4d40a3360 con 0x55c6dc4c20f0 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/3428970294 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb4d409b410 con 0x55c6dc4c20f0 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4efbd5b80 10 client.15972 fetch_fsmap learned FSMap version 30 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/3428970294 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55c6dc3b6940 con 0x55c6dc4c20f0 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/3428970294 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fb4d40a3360 con 0x55c6dc4c20f0 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/3428970294 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fb4d4095bb0 con 0x55c6dc4c20f0 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4efbd5b80 10 client.15972 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4efbd5b80 10 client.15972 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4efbd5b80 20 client.15972 populate_metadata read hostname 'smithi093' 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c6dc525f80 0x55c6dc528360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4efbd5b80 4 client.15972 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/3428970294 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55c6dc4029a0 con 0x55c6dc525f80 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.920+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c6dc525f80 0x55c6dc528360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:36.926 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.921+0000 7fb4c4ff9700 10 client.15972 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:36.927 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:36.921+0000 7fb4e17e3700 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c6dc525f80 0x55c6dc528360 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dafb720ff8132a4 server_cookie=812a0ba35a1e1626 in_seq=0 out_seq=0 2023-12-14T18:32:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:36 smithi160 ceph-mon[43840]: pgmap v187: 129 pgs: 129 active+clean; 722 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 337 op/s 2023-12-14T18:32:37.318 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:36 smithi093 ceph-mon[42070]: pgmap v187: 129 pgs: 129 active+clean; 722 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 337 op/s 2023-12-14T18:32:37.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:36 smithi097 ceph-mon[43486]: pgmap v187: 129 pgs: 129 active+clean; 722 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 3.3 MiB/s wr, 337 op/s 2023-12-14T18:32:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.329+0000 7fb4c4ff9700 1 -- 172.21.15.93:0/3428970294 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55c6dc4029a0 con 0x55c6dc525f80 2023-12-14T18:32:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.329+0000 7fb4c4ff9700 10 client.15972 handle_command_reply: tid=0 2023-12-14T18:32:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.329+0000 7fb4efbd5b80 1 client.15972 shutdown 2023-12-14T18:32:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/3428970294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 msgr2=0x55c6dc4c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:38.331 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c6d20 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fb4d40974a0 tx=0x7fb4d40b93f0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/3428970294 shutdown_connections 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55c6dc525f80 0x55c6dc528360 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55c6dc4c20f0 0x55c6dc4c6d20 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55c6dc314430 0x55c6dc4c62a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4c4ff9700 0 client.15972 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 --2- 172.21.15.93:0/3428970294 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55c6dc4b7370 0x55c6dc4c67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/3428970294 >> 172.21.15.93:0/3428970294 conn(0x55c6dc4ad890 msgr2=0x55c6dc4cdbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/3428970294 shutdown_connections 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 1 -- 172.21.15.93:0/3428970294 wait complete. 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4efbd5b80 20 client.15972 trim_cache size 0 max 16384 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:32:38.332 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:32:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:32:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:32:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb4c80a12c0 msgr2=0x7fb4c80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb4c80a12c0 0x7fb4c80a3770 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fb4cc009c60 tx=0x7fb4cc009260 comp rx=0 tx=0).stop 2023-12-14T18:32:38.333 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 msgr2=0x7fb4dc19bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 0x7fb4dc19bfd0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fb4d4007870 tx=0x7fb4d40078a0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3387183213 shutdown_connections 2023-12-14T18:32:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fb4c80a12c0 0x7fb4c80a3770 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fb4dc104400 0x7fb4dc19bfd0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.330+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fb4dc0ff3c0 0x7fb4dc1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.331+0000 7fb4e3a47700 1 --2- 172.21.15.93:0/3387183213 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fb4dc0ffd60 0x7fb4dc198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.331+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3387183213 >> 172.21.15.93:0/3387183213 conn(0x7fb4dc0763a0 msgr2=0x7fb4dc068960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.331+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3387183213 shutdown_connections 2023-12-14T18:32:38.334 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:38.331+0000 7fb4e3a47700 1 -- 172.21.15.93:0/3387183213 wait complete. 2023-12-14T18:32:39.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:38 smithi160 ceph-mon[43840]: pgmap v188: 129 pgs: 129 active+clean; 722 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 164 op/s 2023-12-14T18:32:39.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:38 smithi093 ceph-mon[42070]: pgmap v188: 129 pgs: 129 active+clean; 722 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 164 op/s 2023-12-14T18:32:39.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:38 smithi097 ceph-mon[43486]: pgmap v188: 129 pgs: 129 active+clean; 722 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.1 MiB/s wr, 164 op/s 2023-12-14T18:32:41.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:40 smithi160 ceph-mon[43840]: pgmap v189: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2023-12-14T18:32:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:40 smithi093 ceph-mon[42070]: pgmap v189: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2023-12-14T18:32:41.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:40 smithi097 ceph-mon[43486]: pgmap v189: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 MiB/s rd, 3.9 MiB/s wr, 421 op/s 2023-12-14T18:32:43.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:42 smithi160 ceph-mon[43840]: pgmap v190: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 372 op/s 2023-12-14T18:32:43.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:42 smithi093 ceph-mon[42070]: pgmap v190: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 372 op/s 2023-12-14T18:32:43.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:42 smithi097 ceph-mon[43486]: pgmap v190: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 372 op/s 2023-12-14T18:32:44.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:43 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:32:44.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:43 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:32:44.360 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:43 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:32:45.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:44 smithi160 ceph-mon[43840]: pgmap v191: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 372 op/s 2023-12-14T18:32:45.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:44 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:32:45.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:44 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:32:45.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:32:45.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:44 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:32:45.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:44 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:32:45.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:44 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:32:45.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:44 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:32:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:44 smithi093 ceph-mon[42070]: pgmap v191: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 372 op/s 2023-12-14T18:32:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:44 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:32:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:44 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:32:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:32:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:44 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:32:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:44 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:32:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:44 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:32:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:44 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:32:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:44 smithi097 ceph-mon[43486]: pgmap v191: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 MiB/s rd, 2.9 MiB/s wr, 372 op/s 2023-12-14T18:32:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:44 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:32:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:44 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:32:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:32:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:44 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:32:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:44 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:32:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:44 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:32:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:44 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:32:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:46 smithi160 ceph-mon[43840]: pgmap v192: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2023-12-14T18:32:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:46 smithi093 ceph-mon[42070]: pgmap v192: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2023-12-14T18:32:47.605 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:46 smithi097 ceph-mon[43486]: pgmap v192: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 MiB/s rd, 3.0 MiB/s wr, 424 op/s 2023-12-14T18:32:49.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:48 smithi160 ceph-mon[43840]: pgmap v193: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2023-12-14T18:32:49.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:48 smithi093 ceph-mon[42070]: pgmap v193: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2023-12-14T18:32:49.502 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:48 smithi097 ceph-mon[43486]: pgmap v193: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2023-12-14T18:32:51.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:50 smithi160 ceph-mon[43840]: pgmap v194: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 355 op/s 2023-12-14T18:32:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:50 smithi093 ceph-mon[42070]: pgmap v194: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 355 op/s 2023-12-14T18:32:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:50 smithi097 ceph-mon[43486]: pgmap v194: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.8 MiB/s wr, 355 op/s 2023-12-14T18:32:51.412 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:32:51.412 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:32:51.452 INFO:tasks.workunit:Running workunit fs/misc/filelock_deadlock.py... 2023-12-14T18:32:51.453 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/filelock_deadlock.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_deadlock.py 2023-12-14T18:32:51.457 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.454+0000 7f505531a700 1 -- 172.21.15.93:0/349846201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 msgr2=0x7f505010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.454+0000 7f505531a700 1 --2- 172.21.15.93:0/349846201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 0x7f505010bfd0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f504800b0d0 tx=0x7f504800b3e0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.454+0000 7f505531a700 1 -- 172.21.15.93:0/349846201 shutdown_connections 2023-12-14T18:32:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.454+0000 7f505531a700 1 --2- 172.21.15.93:0/349846201 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 0x7f505010bfd0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.454+0000 7f505531a700 1 --2- 172.21.15.93:0/349846201 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500ff3c0 0x7f50500ff790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.454+0000 7f505531a700 1 --2- 172.21.15.93:0/349846201 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50500ffd60 0x7f5050103d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.454+0000 7f505531a700 1 -- 172.21.15.93:0/349846201 >> 172.21.15.93:0/349846201 conn(0x7f50500763a0 msgr2=0x7f50500767a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.454+0000 7f505531a700 1 -- 172.21.15.93:0/349846201 shutdown_connections 2023-12-14T18:32:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.454+0000 7f505531a700 1 -- 172.21.15.93:0/349846201 wait complete. 2023-12-14T18:32:51.458 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f505531a700 1 Processor -- start 2023-12-14T18:32:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f505531a700 1 -- start start 2023-12-14T18:32:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f505531a700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500ff3c0 0x7f5050195430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f505531a700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50500ffd60 0x7f5050195970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f505531a700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 0x7f5050199cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f505531a700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505019a300 con 0x7f5050104400 2023-12-14T18:32:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f505531a700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505019a470 con 0x7f50500ff3c0 2023-12-14T18:32:51.459 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f505531a700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f505019a630 con 0x7f50500ffd60 2023-12-14T18:32:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f504f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 0x7f5050199cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f504effd700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500ff3c0 0x7f5050195430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.460 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f504f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 0x7f5050199cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45624/0 (socket says 172.21.15.93:45624) 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.455+0000 7f504f7fe700 1 -- 172.21.15.93:0/292221202 learned_addr learned my addr 172.21.15.93:0/292221202 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f504e7fc700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50500ffd60 0x7f5050195970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f504f7fe700 1 -- 172.21.15.93:0/292221202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50500ffd60 msgr2=0x7f5050195970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f504f7fe700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50500ffd60 0x7f5050195970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f504f7fe700 1 -- 172.21.15.93:0/292221202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500ff3c0 msgr2=0x7f5050195430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f504f7fe700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500ff3c0 0x7f5050195430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f504f7fe700 1 -- 172.21.15.93:0/292221202 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5048009d00 con 0x7f5050104400 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f504f7fe700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 0x7f5050199cc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5048007870 tx=0x7f50480078a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f5037fff700 1 -- 172.21.15.93:0/292221202 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504800e040 con 0x7f5050104400 2023-12-14T18:32:51.461 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f5037fff700 1 -- 172.21.15.93:0/292221202 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5048005070 con 0x7f5050104400 2023-12-14T18:32:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f5037fff700 1 -- 172.21.15.93:0/292221202 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504801b6c0 con 0x7f5050104400 2023-12-14T18:32:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f505531a700 1 -- 172.21.15.93:0/292221202 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f505010f1d0 con 0x7f5050104400 2023-12-14T18:32:51.462 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.456+0000 7f505531a700 1 -- 172.21.15.93:0/292221202 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505010f040 con 0x7f5050104400 2023-12-14T18:32:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.458+0000 7f5037fff700 1 -- 172.21.15.93:0/292221202 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f50480051e0 con 0x7f5050104400 2023-12-14T18:32:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.459+0000 7f5037fff700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f50380a1340 0x7f50380a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.459+0000 7f5037fff700 1 -- 172.21.15.93:0/292221202 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f50480ce0a0 con 0x7f5050104400 2023-12-14T18:32:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.459+0000 7f504effd700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f50380a1340 0x7f50380a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.460+0000 7f504effd700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f50380a1340 0x7f50380a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5044009c60 tx=0x7f5044009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.460+0000 7f50614a8b80 1 Processor -- start 2023-12-14T18:32:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.460+0000 7f50614a8b80 1 -- start start 2023-12-14T18:32:51.470 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.460+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea3452b430 0x55ea346cde30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.460+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346ce7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.460+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346dd6f0 0x55ea346dfad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.460+0000 7f50614a8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea345b8580 con 0x55ea346dd6f0 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.460+0000 7f50614a8b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34627810 con 0x55ea3452b430 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.460+0000 7f50614a8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea34620ca0 con 0x55ea346ce370 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346dd6f0 0x55ea346dfad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504f7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346dd6f0 0x55ea346dfad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45626/0 (socket says 172.21.15.93:45626) 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504f7fe700 1 -- 172.21.15.93:0/4047269993 learned_addr learned my addr 172.21.15.93:0/4047269993 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504effd700 1 --2- 172.21.15.93:0/4047269993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea3452b430 0x55ea346cde30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504e7fc700 1 --2- 172.21.15.93:0/4047269993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346ce7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.471 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504f7fe700 1 -- 172.21.15.93:0/4047269993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 msgr2=0x55ea346ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504f7fe700 1 --2- 172.21.15.93:0/4047269993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346ce7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504f7fe700 1 -- 172.21.15.93:0/4047269993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea3452b430 msgr2=0x55ea346cde30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504f7fe700 1 --2- 172.21.15.93:0/4047269993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea3452b430 0x55ea346cde30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504f7fe700 1 -- 172.21.15.93:0/4047269993 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea344a2350 con 0x55ea346dd6f0 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f504f7fe700 1 --2- 172.21.15.93:0/4047269993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346dd6f0 0x55ea346dfad0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5048097980 tx=0x7f504809ace0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa3235ec04248fa server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f50357fa700 1 -- 172.21.15.93:0/4047269993 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50480054e0 con 0x55ea346dd6f0 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f50357fa700 1 -- 172.21.15.93:0/4047269993 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f504809af10 con 0x55ea346dd6f0 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.461+0000 7f50357fa700 1 -- 172.21.15.93:0/4047269993 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504809a400 con 0x55ea346dd6f0 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.462+0000 7f50614a8b80 1 -- 172.21.15.93:0/4047269993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346dd6f0 msgr2=0x55ea346dfad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.462+0000 7f50614a8b80 1 --2- 172.21.15.93:0/4047269993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346dd6f0 0x55ea346dfad0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5048097980 tx=0x7f504809ace0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.462+0000 7f50614a8b80 1 -- 172.21.15.93:0/4047269993 shutdown_connections 2023-12-14T18:32:51.472 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.462+0000 7f50614a8b80 1 --2- 172.21.15.93:0/4047269993 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346dd6f0 0x55ea346dfad0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.462+0000 7f50614a8b80 1 --2- 172.21.15.93:0/4047269993 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea3452b430 0x55ea346cde30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.462+0000 7f50614a8b80 1 --2- 172.21.15.93:0/4047269993 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346ce7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.462+0000 7f50614a8b80 1 -- 172.21.15.93:0/4047269993 >> 172.21.15.93:0/4047269993 conn(0x55ea346c4890 msgr2=0x55ea346ca200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.462+0000 7f50614a8b80 1 -- 172.21.15.93:0/4047269993 shutdown_connections 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.462+0000 7f50614a8b80 1 -- 172.21.15.93:0/4047269993 wait complete. 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f50614a8b80 1 -- start start 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f50614a8b80 1 Processor -- start 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea3452b430 0x55ea346ddac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346ce370 0x55ea346de000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea346de540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.473 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f50614a8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea345bcf30 con 0x55ea346ce370 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f50614a8b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea345bcbd0 con 0x55ea346dd6f0 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f50614a8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea344a4640 con 0x55ea3452b430 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f50614a8b80 10 client.0 _ll_register_callbacks cb 0x55ea345e08b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea3452b430 0x55ea346ddac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504effd700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea3452b430 0x55ea346ddac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.160:3300/0 says I am v2:172.21.15.93:47444/0 (socket says 172.21.15.93:47444) 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504effd700 1 -- 172.21.15.93:0/1582641863 learned_addr learned my addr 172.21.15.93:0/1582641863 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504e7fc700 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346ce370 0x55ea346de000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504f7fe700 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea346de540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504effd700 1 -- 172.21.15.93:0/1582641863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 msgr2=0x55ea346de540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504effd700 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea346de540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504effd700 1 -- 172.21.15.93:0/1582641863 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346ce370 msgr2=0x55ea346de000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:51.474 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504effd700 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346ce370 0x55ea346de000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.463+0000 7f504effd700 1 -- 172.21.15.93:0/1582641863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f505010f040 con 0x55ea3452b430 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f504effd700 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea3452b430 0x55ea346ddac0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5044003900 tx=0x7f50440047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=952c0727a665ac79 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f5032ff5700 10 client.0 ms_handle_connect on v2:172.21.15.160:3300/0 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f5032ff5700 1 -- 172.21.15.93:0/1582641863 <== mon.1 v2:172.21.15.160:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044022070 con 0x55ea3452b430 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f50614a8b80 1 -- 172.21.15.93:0/1582641863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55ea346c1ea0 con 0x55ea3452b430 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f5032ff5700 1 -- 172.21.15.93:0/1582641863 <== mon.1 v2:172.21.15.160:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f504401d620 con 0x55ea3452b430 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f5032ff5700 1 -- 172.21.15.93:0/1582641863 <== mon.1 v2:172.21.15.160:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f504401d7e0 con 0x55ea3452b430 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f50614a8b80 10 client.25603 fetch_fsmap learned FSMap version 30 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f50614a8b80 1 -- 172.21.15.93:0/1582641863 --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ea345e7110 con 0x55ea3452b430 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f5032ff5700 1 -- 172.21.15.93:0/1582641863 <== mon.1 v2:172.21.15.160:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55ea346c1ea0 con 0x55ea3452b430 2023-12-14T18:32:51.475 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f5032ff5700 1 -- 172.21.15.93:0/1582641863 <== mon.1 v2:172.21.15.160:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5044027a00 con 0x55ea3452b430 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f50614a8b80 10 client.25603 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f50614a8b80 10 client.25603 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f50614a8b80 20 client.25603 populate_metadata read hostname 'smithi093' 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ea346ecda0 0x55ea346ef1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f50614a8b80 4 client.25603 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.464+0000 7f50614a8b80 1 -- 172.21.15.93:0/1582641863 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55ea34619c80 con 0x55ea346ecda0 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.465+0000 7f504e7fc700 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ea346ecda0 0x55ea346ef1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.465+0000 7f5032ff5700 10 client.25603 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.465+0000 7f504e7fc700 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ea346ecda0 0x55ea346ef1b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c70c412ccf08072 server_cookie=62161f652c776f0d in_seq=0 out_seq=0 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f5032ff5700 1 -- 172.21.15.93:0/1582641863 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55ea34619c80 con 0x55ea346ecda0 2023-12-14T18:32:51.476 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f5032ff5700 10 client.25603 handle_command_reply: tid=0 2023-12-14T18:32:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 client.25603 shutdown 2023-12-14T18:32:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 -- 172.21.15.93:0/1582641863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea3452b430 msgr2=0x55ea346ddac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea3452b430 0x55ea346ddac0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5044003900 tx=0x7f50440047c0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 -- 172.21.15.93:0/1582641863 shutdown_connections 2023-12-14T18:32:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ea346ecda0 0x55ea346ef1b0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea346de540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea3452b430 0x55ea346ddac0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f5032ff5700 0 client.25603 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:51.477 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1582641863 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea346ce370 0x55ea346de000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 -- 172.21.15.93:0/1582641863 >> 172.21.15.93:0/1582641863 conn(0x55ea346c4890 msgr2=0x55ea346c9d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 -- 172.21.15.93:0/1582641863 shutdown_connections 2023-12-14T18:32:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 1 -- 172.21.15.93:0/1582641863 wait complete. 2023-12-14T18:32:51.478 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.466+0000 7f50614a8b80 20 client.25603 trim_cache size 0 max 16384 2023-12-14T18:32:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.481+0000 7f50614a8b80 1 Processor -- start 2023-12-14T18:32:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f50614a8b80 1 -- start start 2023-12-14T18:32:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346e2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346e8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.489 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea346dbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f50614a8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea346dcc20 con 0x55ea3452b430 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f50614a8b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea346daca0 con 0x55ea346dd6f0 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f50614a8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea346d8360 con 0x55ea346ce370 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346e2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346e2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45646/0 (socket says 172.21.15.93:45646) 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504effd700 1 -- 172.21.15.93:0/500338190 learned_addr learned my addr 172.21.15.93:0/500338190 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504f7fe700 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea346dbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504e7fc700 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346e8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504effd700 1 -- 172.21.15.93:0/500338190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 msgr2=0x55ea346e8280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504effd700 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346e8280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504effd700 1 -- 172.21.15.93:0/500338190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 msgr2=0x55ea346dbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:51.490 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504effd700 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea346dbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504effd700 1 -- 172.21.15.93:0/500338190 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55ea346dd090 con 0x55ea3452b430 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504f7fe700 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea346dbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504e7fc700 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f504effd700 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346e2ac0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5044009fd0 tx=0x7f50440092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=482bd62fcb4788db server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.482+0000 7f5035ffb700 1 -- 172.21.15.93:0/500338190 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044022030 con 0x55ea3452b430 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f5035ffb700 1 -- 172.21.15.93:0/500338190 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f5044004560 con 0x55ea3452b430 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f5035ffb700 1 -- 172.21.15.93:0/500338190 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f504401dcc0 con 0x55ea3452b430 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f50614a8b80 1 -- 172.21.15.93:0/500338190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 msgr2=0x55ea346e2ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f50614a8b80 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346e2ac0 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5044009fd0 tx=0x7f50440092d0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f50614a8b80 1 -- 172.21.15.93:0/500338190 shutdown_connections 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f50614a8b80 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346e2ac0 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f50614a8b80 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea346dbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.491 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f50614a8b80 1 --2- 172.21.15.93:0/500338190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346e8280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f50614a8b80 1 -- 172.21.15.93:0/500338190 >> 172.21.15.93:0/500338190 conn(0x55ea346c4890 msgr2=0x55ea346c8930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f50614a8b80 1 -- 172.21.15.93:0/500338190 shutdown_connections 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.483+0000 7f50614a8b80 1 -- 172.21.15.93:0/500338190 wait complete. 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f50614a8b80 1 -- start start 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f50614a8b80 1 Processor -- start 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346d9920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346d9e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f50614a8b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea34739c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f50614a8b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3473a1d0 con 0x55ea3452b430 2023-12-14T18:32:51.492 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f50614a8b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3473a310 con 0x55ea346dd6f0 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f50614a8b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55ea3473a4d0 con 0x55ea346ce370 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f50614a8b80 10 client.0 _ll_register_callbacks cb 0x7f50440043e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346d9920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504effd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346d9920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45660/0 (socket says 172.21.15.93:45660) 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504effd700 1 -- 172.21.15.93:0/1086249697 learned_addr learned my addr 172.21.15.93:0/1086249697 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504f7fe700 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea34739c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504e7fc700 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346d9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504effd700 1 -- 172.21.15.93:0/1086249697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 msgr2=0x55ea346d9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504effd700 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346d9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504effd700 1 -- 172.21.15.93:0/1086249697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 msgr2=0x55ea34739c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:32:51.493 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504effd700 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea34739c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504effd700 1 -- 172.21.15.93:0/1086249697 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55ea344a2350 con 0x55ea3452b430 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504f7fe700 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea34739c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f504effd700 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346d9920 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5044005950 tx=0x7f504401de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=359bf1eecbe1f660 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.484+0000 7f5032ff5700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f5032ff5700 1 -- 172.21.15.93:0/1086249697 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5044022040 con 0x55ea3452b430 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f5032ff5700 1 -- 172.21.15.93:0/1086249697 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7f5044023070 con 0x55ea3452b430 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f50614a8b80 1 -- 172.21.15.93:0/1086249697 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f504809af10 con 0x55ea3452b430 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f5032ff5700 1 -- 172.21.15.93:0/1086249697 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5044005410 con 0x55ea3452b430 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f50614a8b80 10 client.15990 fetch_fsmap learned FSMap version 30 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f50614a8b80 1 -- 172.21.15.93:0/1086249697 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55ea345e7110 con 0x55ea3452b430 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f5032ff5700 1 -- 172.21.15.93:0/1086249697 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f504809af10 con 0x55ea3452b430 2023-12-14T18:32:51.494 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f5032ff5700 1 -- 172.21.15.93:0/1086249697 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7f5044004890 con 0x55ea3452b430 2023-12-14T18:32:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f50614a8b80 10 client.15990 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:32:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f50614a8b80 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:32:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f50614a8b80 20 client.15990 populate_metadata read hostname 'smithi093' 2023-12-14T18:32:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ea346ecda0 0x55ea34743920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:32:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f50614a8b80 4 client.15990 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:32:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f50614a8b80 1 -- 172.21.15.93:0/1086249697 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55ea346199a0 con 0x55ea346ecda0 2023-12-14T18:32:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.485+0000 7f504e7fc700 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ea346ecda0 0x55ea34743920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:32:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.486+0000 7f5032ff5700 10 client.15990 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:51.495 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.486+0000 7f504e7fc700 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ea346ecda0 0x55ea34743920 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a52d1890529d803e server_cookie=8ef4005a2eb88631 in_seq=0 out_seq=0 2023-12-14T18:32:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f5032ff5700 1 -- 172.21.15.93:0/1086249697 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55ea346199a0 con 0x55ea346ecda0 2023-12-14T18:32:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f5032ff5700 10 client.15990 handle_command_reply: tid=0 2023-12-14T18:32:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 client.15990 shutdown 2023-12-14T18:32:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 -- 172.21.15.93:0/1086249697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 msgr2=0x55ea346d9920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346d9920 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f5044005950 tx=0x7f504401de40 comp rx=0 tx=0).stop 2023-12-14T18:32:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 -- 172.21.15.93:0/1086249697 shutdown_connections 2023-12-14T18:32:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55ea346ecda0 0x55ea34743920 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55ea3452b430 0x55ea346d9920 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.589 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f5032ff5700 0 client.15990 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:32:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55ea346dd6f0 0x55ea34739c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 --2- 172.21.15.93:0/1086249697 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55ea346ce370 0x55ea346d9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 -- 172.21.15.93:0/1086249697 >> 172.21.15.93:0/1086249697 conn(0x55ea346c4890 msgr2=0x55ea346dbfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 -- 172.21.15.93:0/1086249697 shutdown_connections 2023-12-14T18:32:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 1 -- 172.21.15.93:0/1086249697 wait complete. 2023-12-14T18:32:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.586+0000 7f50614a8b80 20 client.15990 trim_cache size 0 max 16384 2023-12-14T18:32:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:32:51.590 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:32:51.591 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 -- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f50380a1340 msgr2=0x7f50380a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f50380a1340 0x7f50380a37f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f5044009c60 tx=0x7f5044009260 comp rx=0 tx=0).stop 2023-12-14T18:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 -- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 msgr2=0x7f5050199cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 0x7f5050199cc0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f5048007870 tx=0x7f50480078a0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 -- 172.21.15.93:0/292221202 shutdown_connections 2023-12-14T18:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f50380a1340 0x7f50380a37f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.592 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f5050104400 0x7f5050199cc0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f50500ff3c0 0x7f5050195430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 --2- 172.21.15.93:0/292221202 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f50500ffd60 0x7f5050195970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 -- 172.21.15.93:0/292221202 >> 172.21.15.93:0/292221202 conn(0x7f50500763a0 msgr2=0x7f50500fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 -- 172.21.15.93:0/292221202 shutdown_connections 2023-12-14T18:32:51.593 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:32:51.587+0000 7f505531a700 1 -- 172.21.15.93:0/292221202 wait complete. 2023-12-14T18:32:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:52 smithi160 ceph-mon[43840]: pgmap v195: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 34 KiB/s wr, 98 op/s 2023-12-14T18:32:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:52 smithi093 ceph-mon[42070]: pgmap v195: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 34 KiB/s wr, 98 op/s 2023-12-14T18:32:53.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:52 smithi097 ceph-mon[43486]: pgmap v195: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 34 KiB/s wr, 98 op/s 2023-12-14T18:32:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:54 smithi160 ceph-mon[43840]: pgmap v196: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 34 KiB/s wr, 98 op/s 2023-12-14T18:32:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:54 smithi093 ceph-mon[42070]: pgmap v196: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 34 KiB/s wr, 98 op/s 2023-12-14T18:32:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:54 smithi097 ceph-mon[43486]: pgmap v196: 129 pgs: 129 active+clean; 738 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 738 KiB/s rd, 34 KiB/s wr, 98 op/s 2023-12-14T18:32:57.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:56 smithi160 ceph-mon[43840]: pgmap v197: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 105 KiB/s wr, 198 op/s 2023-12-14T18:32:57.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:56 smithi093 ceph-mon[42070]: pgmap v197: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 105 KiB/s wr, 198 op/s 2023-12-14T18:32:57.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:56 smithi097 ceph-mon[43486]: pgmap v197: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 902 KiB/s rd, 105 KiB/s wr, 198 op/s 2023-12-14T18:32:59.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:32:58 smithi160 ceph-mon[43840]: pgmap v198: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 71 KiB/s wr, 146 op/s 2023-12-14T18:32:59.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:32:58 smithi093 ceph-mon[42070]: pgmap v198: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 71 KiB/s wr, 146 op/s 2023-12-14T18:32:59.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:32:58 smithi097 ceph-mon[43486]: pgmap v198: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 433 KiB/s rd, 71 KiB/s wr, 146 op/s 2023-12-14T18:33:01.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:00 smithi160 ceph-mon[43840]: pgmap v199: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 72 KiB/s wr, 150 op/s 2023-12-14T18:33:01.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:00 smithi093 ceph-mon[42070]: pgmap v199: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 72 KiB/s wr, 150 op/s 2023-12-14T18:33:01.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:00 smithi097 ceph-mon[43486]: pgmap v199: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 434 KiB/s rd, 72 KiB/s wr, 150 op/s 2023-12-14T18:33:01.583 INFO:tasks.workunit.client.0.smithi093.stdout:ok 2023-12-14T18:33:01.588 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:33:01.588 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:33:01.683 INFO:tasks.workunit:Running workunit fs/misc/filelock_interrupt.py... 2023-12-14T18:33:01.684 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/filelock_interrupt.py> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/filelock_interrupt.py 2023-12-14T18:33:03.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:02 smithi160 ceph-mon[43840]: pgmap v200: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 71 KiB/s wr, 103 op/s 2023-12-14T18:33:03.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:02 smithi093 ceph-mon[42070]: pgmap v200: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 71 KiB/s wr, 103 op/s 2023-12-14T18:33:03.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:02 smithi097 ceph-mon[43486]: pgmap v200: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 71 KiB/s wr, 103 op/s 2023-12-14T18:33:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.707+0000 7fbdc8798700 1 -- 172.21.15.93:0/965740798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc01043f0 msgr2=0x7fbdc010bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.707+0000 7fbdc8798700 1 --2- 172.21.15.93:0/965740798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc01043f0 0x7fbdc010bfc0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7fbdb800b0d0 tx=0x7fbdb800b3e0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.707+0000 7fbdc8798700 1 -- 172.21.15.93:0/965740798 shutdown_connections 2023-12-14T18:33:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.707+0000 7fbdc8798700 1 --2- 172.21.15.93:0/965740798 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc01043f0 0x7fbdc010bfc0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.712 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.707+0000 7fbdc8798700 1 --2- 172.21.15.93:0/965740798 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc00ff3b0 0x7fbdc00ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.707+0000 7fbdc8798700 1 --2- 172.21.15.93:0/965740798 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc00ffd50 0x7fbdc0103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.707+0000 7fbdc8798700 1 -- 172.21.15.93:0/965740798 >> 172.21.15.93:0/965740798 conn(0x7fbdc0075f70 msgr2=0x7fbdc0076370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.707+0000 7fbdc8798700 1 -- 172.21.15.93:0/965740798 shutdown_connections 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.707+0000 7fbdc8798700 1 -- 172.21.15.93:0/965740798 wait complete. 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc8798700 1 Processor -- start 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc8798700 1 -- start start 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc8798700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc00ff3b0 0x7fbdc0197280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc8798700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc00ffd50 0x7fbdc01977c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc8798700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc01043f0 0x7fbdc01a2300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc8798700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc0198100 con 0x7fbdc00ff3b0 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc8798700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc01a2840 con 0x7fbdc00ffd50 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc8798700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc01a2a00 con 0x7fbdc01043f0 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6534700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc00ff3b0 0x7fbdc0197280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.713 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6534700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc00ff3b0 0x7fbdc0197280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49220/0 (socket says 172.21.15.93:49220) 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6534700 1 -- 172.21.15.93:0/2258405242 learned_addr learned my addr 172.21.15.93:0/2258405242 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6d35700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc01043f0 0x7fbdc01a2300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc00ffd50 0x7fbdc01977c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6534700 1 -- 172.21.15.93:0/2258405242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc01043f0 msgr2=0x7fbdc01a2300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6534700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc01043f0 0x7fbdc01a2300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6534700 1 -- 172.21.15.93:0/2258405242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc00ffd50 msgr2=0x7fbdc01977c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6534700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc00ffd50 0x7fbdc01977c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6534700 1 -- 172.21.15.93:0/2258405242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8009d00 con 0x7fbdc00ff3b0 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc6d35700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc01043f0 0x7fbdc01a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.708+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc00ffd50 0x7fbdc01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.709+0000 7fbdc6534700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc00ff3b0 0x7fbdc0197280 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00d950 tx=0x7fbdbc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.709+0000 7fbdb37fe700 1 -- 172.21.15.93:0/2258405242 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc009b40 con 0x7fbdc00ff3b0 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.709+0000 7fbdc8798700 1 -- 172.21.15.93:0/2258405242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc010f1d0 con 0x7fbdc00ff3b0 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.709+0000 7fbdb37fe700 1 -- 172.21.15.93:0/2258405242 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdbc009ca0 con 0x7fbdc00ff3b0 2023-12-14T18:33:04.714 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.709+0000 7fbdb37fe700 1 -- 172.21.15.93:0/2258405242 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc00f620 con 0x7fbdc00ff3b0 2023-12-14T18:33:04.715 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.709+0000 7fbdc8798700 1 -- 172.21.15.93:0/2258405242 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc010f040 con 0x7fbdc00ff3b0 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.711+0000 7fbdb37fe700 1 -- 172.21.15.93:0/2258405242 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbdbc010460 con 0x7fbdc00ff3b0 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.712+0000 7fbdb37fe700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbdac0a12c0 0x7fbdac0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.712+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbdac0a12c0 0x7fbdac0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.712+0000 7fbdb37fe700 1 -- 172.21.15.93:0/2258405242 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fbdbc0cd3f0 con 0x7fbdc00ff3b0 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.713+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbdac0a12c0 0x7fbdac0a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4005e10 tx=0x7fbdb4005d20 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.713+0000 7fbdd4926b80 1 Processor -- start 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.713+0000 7fbdd4926b80 1 -- start start 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327ae9e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327aea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327af74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3279d4580 con 0x55a327aea370 2023-12-14T18:33:04.720 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a327a43810 con 0x55a327af50f0 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a327a3cca0 con 0x55a327947430 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc5d33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327aea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc5d33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327aea7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49222/0 (socket says 172.21.15.93:49222) 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc5d33700 1 -- 172.21.15.93:0/1505193281 learned_addr learned my addr 172.21.15.93:0/1505193281 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc6d35700 1 --2- 172.21.15.93:0/1505193281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327af74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc6534700 1 --2- 172.21.15.93:0/1505193281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327ae9e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc5d33700 1 -- 172.21.15.93:0/1505193281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 msgr2=0x55a327ae9e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/1505193281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327ae9e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc5d33700 1 -- 172.21.15.93:0/1505193281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 msgr2=0x55a327af74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/1505193281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327af74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc5d33700 1 -- 172.21.15.93:0/1505193281 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a3278be350 con 0x55a327aea370 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/1505193281 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327aea7e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400ec80 tx=0x7fbdb4011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be08cb4f6e974399 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdb0ff9700 1 -- 172.21.15.93:0/1505193281 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4022070 con 0x55a327aea370 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdb0ff9700 1 -- 172.21.15.93:0/1505193281 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdb40044e0 con 0x55a327aea370 2023-12-14T18:33:04.721 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.714+0000 7fbdb0ff9700 1 -- 172.21.15.93:0/1505193281 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4004c20 con 0x55a327aea370 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1505193281 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 msgr2=0x55a327aea7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1505193281 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327aea7e0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fbdb400ec80 tx=0x7fbdb4011000 comp rx=0 tx=0).stop 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1505193281 shutdown_connections 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1505193281 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327af74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1505193281 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327ae9e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1505193281 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327aea7e0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1505193281 >> 172.21.15.93:0/1505193281 conn(0x55a327ae0890 msgr2=0x55a327ae6200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1505193281 shutdown_connections 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1505193281 wait complete. 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.715+0000 7fbdd4926b80 1 -- start start 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdd4926b80 1 Processor -- start 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327af49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327af2250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3279d8f30 con 0x55a327aea370 2023-12-14T18:33:04.722 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3279d8bd0 con 0x55a327af50f0 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a3278c0640 con 0x55a327947430 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdd4926b80 10 client.0 _ll_register_callbacks cb 0x55a3279fc8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc5d33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc5d33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af1cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49234/0 (socket says 172.21.15.93:49234) 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc5d33700 1 -- 172.21.15.93:0/3062081377 learned_addr learned my addr 172.21.15.93:0/3062081377 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc6d35700 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327af2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc6534700 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327af49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc5d33700 1 -- 172.21.15.93:0/3062081377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 msgr2=0x55a327af49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327af49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc5d33700 1 -- 172.21.15.93:0/3062081377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 msgr2=0x55a327af2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327af2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.716+0000 7fbdc5d33700 1 -- 172.21.15.93:0/3062081377 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdb400e960 con 0x55a327aea370 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af1cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4011fd0 tx=0x7fbdb4011e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3950defcf37db0e server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbda9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:33:04.723 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbda9ffb700 1 -- 172.21.15.93:0/3062081377 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4022070 con 0x55a327aea370 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbda9ffb700 1 -- 172.21.15.93:0/3062081377 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fbdb401eab0 con 0x55a327aea370 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdd4926b80 1 -- 172.21.15.93:0/3062081377 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a327addea0 con 0x55a327aea370 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbda9ffb700 1 -- 172.21.15.93:0/3062081377 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdb401ec70 con 0x55a327aea370 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdd4926b80 10 client.16002 fetch_fsmap learned FSMap version 30 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdd4926b80 1 -- 172.21.15.93:0/3062081377 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a327a03110 con 0x55a327aea370 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbda9ffb700 1 -- 172.21.15.93:0/3062081377 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x55a327addea0 con 0x55a327aea370 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbda9ffb700 1 -- 172.21.15.93:0/3062081377 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fbdb4010460 con 0x55a327aea370 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdd4926b80 10 client.16002 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdd4926b80 10 client.16002 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdd4926b80 20 client.16002 populate_metadata read hostname 'smithi093' 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55a327b04ae0 0x55a327b06ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdd4926b80 4 client.16002 mds_command: new command op to 24461 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.717+0000 7fbdd4926b80 1 -- 172.21.15.93:0/3062081377 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a327a35c80 con 0x55a327b04ae0 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.718+0000 7fbdc6534700 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55a327b04ae0 0x55a327b06ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.718+0000 7fbdc6534700 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55a327b04ae0 0x55a327b06ef0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fea549b8308596d server_cookie=5bedc3683dba5a9a in_seq=0 out_seq=0 2023-12-14T18:33:04.724 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.718+0000 7fbda9ffb700 10 client.16002 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:33:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbda9ffb700 1 -- 172.21.15.93:0/3062081377 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+18291 (crc 0 0 0) 0x55a327a35c80 con 0x55a327b04ae0 2023-12-14T18:33:04.725 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbda9ffb700 10 client.16002 handle_command_reply: tid=0 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 client.16002 shutdown 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 -- 172.21.15.93:0/3062081377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 msgr2=0x55a327af1cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af1cf0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4011fd0 tx=0x7fbdb4011e80 comp rx=0 tx=0).stop 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 -- 172.21.15.93:0/3062081377 shutdown_connections 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55a327b04ae0 0x55a327b06ef0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327af2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327af49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbda9ffb700 0 client.16002 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/3062081377 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af1cf0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 -- 172.21.15.93:0/3062081377 >> 172.21.15.93:0/3062081377 conn(0x55a327ae0890 msgr2=0x55a327ae5d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 -- 172.21.15.93:0/3062081377 shutdown_connections 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 1 -- 172.21.15.93:0/3062081377 wait complete. 2023-12-14T18:33:04.726 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.719+0000 7fbdd4926b80 20 client.16002 trim_cache size 0 max 16384 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdd4926b80 1 Processor -- start 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdd4926b80 1 -- start start 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327afbc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af4500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327b00b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a327aebd00 con 0x55a327aea370 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a327aea740 con 0x55a327af50f0 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a327af7a10 con 0x55a327947430 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc5d33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc5d33700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af4500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49244/0 (socket says 172.21.15.93:49244) 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc5d33700 1 -- 172.21.15.93:0/2351712428 learned_addr learned my addr 172.21.15.93:0/2351712428 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc6d35700 1 --2- 172.21.15.93:0/2351712428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327b00b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.740 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc6534700 1 --2- 172.21.15.93:0/2351712428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327afbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc5d33700 1 -- 172.21.15.93:0/2351712428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 msgr2=0x55a327afbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/2351712428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327afbc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc5d33700 1 -- 172.21.15.93:0/2351712428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 msgr2=0x55a327b00b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/2351712428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327b00b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc5d33700 1 -- 172.21.15.93:0/2351712428 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a327a02e60 con 0x55a327aea370 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.734+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/2351712428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af4500 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbdb401efd0 tx=0x7fbdb4005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba0fa7f7cbcf52 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdb17fa700 1 -- 172.21.15.93:0/2351712428 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb4022070 con 0x55a327aea370 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdb17fa700 1 -- 172.21.15.93:0/2351712428 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdb4023040 con 0x55a327aea370 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdb17fa700 1 -- 172.21.15.93:0/2351712428 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb401ed30 con 0x55a327aea370 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 -- 172.21.15.93:0/2351712428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 msgr2=0x55a327af4500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/2351712428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af4500 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fbdb401efd0 tx=0x7fbdb4005b40 comp rx=0 tx=0).stop 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 -- 172.21.15.93:0/2351712428 shutdown_connections 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/2351712428 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327af50f0 0x55a327b00b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/2351712428 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327947430 0x55a327afbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/2351712428 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327aea370 0x55a327af4500 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.741 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 -- 172.21.15.93:0/2351712428 >> 172.21.15.93:0/2351712428 conn(0x55a327ae0890 msgr2=0x55a327b00430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 -- 172.21.15.93:0/2351712428 shutdown_connections 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 -- 172.21.15.93:0/2351712428 wait complete. 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.735+0000 7fbdd4926b80 1 -- start start 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdd4926b80 1 Processor -- start 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327947430 0x55a327b4c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327aea370 0x55a327b4ce30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdd4926b80 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327af50f0 0x55a327b57970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a327b4d700 con 0x55a327947430 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a327b57eb0 con 0x55a327aea370 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdd4926b80 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a327b57ff0 con 0x55a327af50f0 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdd4926b80 10 client.0 _ll_register_callbacks cb 0x7fbdb400eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6534700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327947430 0x55a327b4c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6534700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327947430 0x55a327b4c8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:49254/0 (socket says 172.21.15.93:49254) 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6534700 1 -- 172.21.15.93:0/1196890006 learned_addr learned my addr 172.21.15.93:0/1196890006 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:33:04.742 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327aea370 0x55a327b4ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6d35700 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327af50f0 0x55a327b57970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6534700 1 -- 172.21.15.93:0/1196890006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327af50f0 msgr2=0x55a327b57970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6534700 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327af50f0 0x55a327b57970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6534700 1 -- 172.21.15.93:0/1196890006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327aea370 msgr2=0x55a327b4ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6534700 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327aea370 0x55a327b4ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6534700 1 -- 172.21.15.93:0/1196890006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a327a03110 con 0x55a327947430 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327aea370 0x55a327b4ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbdc6534700 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327947430 0x55a327b4c8f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc000c00 tx=0x7fbdbc010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a430e91c7d6f6bc server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbda9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.93:3300/0 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbda9ffb700 1 -- 172.21.15.93:0/1196890006 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdbc0caa70 con 0x55a327947430 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.736+0000 7fbda9ffb700 1 -- 172.21.15.93:0/1196890006 <== mon.0 v2:172.21.15.93:3300/0 2 ==== osd_map(87..87 src has 1..87) v4 ==== 9278+0+0 (secure 0 0 0) 0x7fbdbc0cabd0 con 0x55a327947430 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1196890006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7fbdb40044e0 con 0x55a327947430 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbda9ffb700 1 -- 172.21.15.93:0/1196890006 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbdbc010710 con 0x55a327947430 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdd4926b80 10 client.16008 fetch_fsmap learned FSMap version 30 2023-12-14T18:33:04.743 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1196890006 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) v3 -- 0x55a3279e9940 con 0x55a327947430 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbda9ffb700 1 -- 172.21.15.93:0/1196890006 <== mon.0 v2:172.21.15.93:3300/0 4 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbdbc0c8080 con 0x55a327947430 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbda9ffb700 1 -- 172.21.15.93:0/1196890006 <== mon.0 v2:172.21.15.93:3300/0 5 ==== fsmap(e 30) v1 ==== 7494+0+0 (secure 0 0 0) 0x7fbdbc099660 con 0x55a327947430 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdd4926b80 10 client.16008 fetch_fsmap finished waiting for FSMap version 30 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdd4926b80 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdd4926b80 20 client.16008 populate_metadata read hostname 'smithi093' 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55a327b59750 0x55a327b5bb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdd4926b80 4 client.16008 mds_command: new command op to 24461 tid=0[{"prefix": "flush journal"}] 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1196890006 --> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] -- command(tid 0: {"prefix": "flush journal"}) v1 -- 0x55a327a359a0 con 0x55a327b59750 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.737+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55a327b59750 0x55a327b5bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.738+0000 7fbdc5d33700 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55a327b59750 0x55a327b5bb60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aad0a973270990a5 server_cookie=c7cae22f92670da8 in_seq=0 out_seq=0 2023-12-14T18:33:04.744 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.738+0000 7fbda9ffb700 10 client.16008 ms_handle_connect on v2:172.21.15.97:6838/3753662808 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.843+0000 7fbda9ffb700 1 -- 172.21.15.93:0/1196890006 <== mds.0 v2:172.21.15.97:6838/3753662808 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+44 (crc 0 0 0) 0x55a327a359a0 con 0x55a327b59750 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.843+0000 7fbda9ffb700 10 client.16008 handle_command_reply: tid=0 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.843+0000 7fbdd4926b80 1 client.16008 shutdown 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.843+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1196890006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327947430 msgr2=0x55a327b4c8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.843+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327947430 0x55a327b4c8f0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc000c00 tx=0x7fbdbc010c40 comp rx=0 tx=0).stop 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.843+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1196890006 shutdown_connections 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.843+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.97:6838/3753662808,v1:172.21.15.97:6839/3753662808] conn(0x55a327b59750 0x55a327b5bb60 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x55a327947430 0x55a327b4c8f0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbda9ffb700 0 client.16008 ms_handle_reset on v2:172.21.15.97:6838/3753662808 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x55a327af50f0 0x55a327b57970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdd4926b80 1 --2- 172.21.15.93:0/1196890006 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x55a327aea370 0x55a327b4ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.846 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1196890006 >> 172.21.15.93:0/1196890006 conn(0x55a327ae0890 msgr2=0x55a327b00cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1196890006 shutdown_connections 2023-12-14T18:33:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdd4926b80 1 -- 172.21.15.93:0/1196890006 wait complete. 2023-12-14T18:33:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdd4926b80 20 client.16008 trim_cache size 0 max 16384 2023-12-14T18:33:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:{ 2023-12-14T18:33:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "message": "", 2023-12-14T18:33:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout: "return_code": 0 2023-12-14T18:33:04.847 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stdout:} 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 -- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbdac0a12c0 msgr2=0x7fbdac0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbdac0a12c0 0x7fbdac0a3770 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fbdb4005e10 tx=0x7fbdb4005d20 comp rx=0 tx=0).stop 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 -- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc00ff3b0 msgr2=0x7fbdc0197280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc00ff3b0 0x7fbdc0197280 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fbdbc00d950 tx=0x7fbdbc00dd10 comp rx=0 tx=0).stop 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 -- 172.21.15.93:0/2258405242 shutdown_connections 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbdac0a12c0 0x7fbdac0a3770 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbdc00ff3b0 0x7fbdc0197280 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbdc01043f0 0x7fbdc01a2300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 --2- 172.21.15.93:0/2258405242 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbdc00ffd50 0x7fbdc01977c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:33:04.848 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 -- 172.21.15.93:0/2258405242 >> 172.21.15.93:0/2258405242 conn(0x7fbdc0075f70 msgr2=0x7fbdc0102cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:33:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.844+0000 7fbdc8798700 1 -- 172.21.15.93:0/2258405242 shutdown_connections 2023-12-14T18:33:04.849 INFO:teuthology.task.background_exec.ubuntu@smithi093.front.sepia.ceph.com.smithi093.stderr:2023-12-14T18:33:04.845+0000 7fbdc8798700 1 -- 172.21.15.93:0/2258405242 wait complete. 2023-12-14T18:33:05.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:04 smithi160 ceph-mon[43840]: pgmap v201: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 71 KiB/s wr, 103 op/s 2023-12-14T18:33:05.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:04 smithi093 ceph-mon[42070]: pgmap v201: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 71 KiB/s wr, 103 op/s 2023-12-14T18:33:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:04 smithi097 ceph-mon[43486]: pgmap v201: 129 pgs: 129 active+clean; 735 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 165 KiB/s rd, 71 KiB/s wr, 103 op/s 2023-12-14T18:33:07.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:06 smithi160 ceph-mon[43840]: pgmap v202: 129 pgs: 129 active+clean; 736 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 153 KiB/s wr, 121 op/s 2023-12-14T18:33:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:06 smithi093 ceph-mon[42070]: pgmap v202: 129 pgs: 129 active+clean; 736 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 153 KiB/s wr, 121 op/s 2023-12-14T18:33:07.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:06 smithi097 ceph-mon[43486]: pgmap v202: 129 pgs: 129 active+clean; 736 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 212 KiB/s rd, 153 KiB/s wr, 121 op/s 2023-12-14T18:33:08.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:07 smithi093 ceph-mon[42070]: osdmap e88: 12 total, 12 up, 12 in 2023-12-14T18:33:08.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:07 smithi097 ceph-mon[43486]: osdmap e88: 12 total, 12 up, 12 in 2023-12-14T18:33:08.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:07 smithi160 ceph-mon[43840]: osdmap e88: 12 total, 12 up, 12 in 2023-12-14T18:33:09.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:08 smithi093 ceph-mon[42070]: pgmap v204: 129 pgs: 129 active+clean; 736 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 98 KiB/s wr, 25 op/s 2023-12-14T18:33:09.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:08 smithi097 ceph-mon[43486]: pgmap v204: 129 pgs: 129 active+clean; 736 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 98 KiB/s wr, 25 op/s 2023-12-14T18:33:09.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:08 smithi160 ceph-mon[43840]: pgmap v204: 129 pgs: 129 active+clean; 736 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 98 KiB/s wr, 25 op/s 2023-12-14T18:33:11.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:11 smithi093 ceph-mon[42070]: pgmap v205: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 111 KiB/s wr, 120 op/s 2023-12-14T18:33:11.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:11 smithi093 ceph-mon[42070]: osdmap e89: 12 total, 12 up, 12 in 2023-12-14T18:33:11.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:11 smithi097 ceph-mon[43486]: pgmap v205: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 111 KiB/s wr, 120 op/s 2023-12-14T18:33:11.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:11 smithi097 ceph-mon[43486]: osdmap e89: 12 total, 12 up, 12 in 2023-12-14T18:33:11.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:11 smithi160 ceph-mon[43840]: pgmap v205: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 111 KiB/s wr, 120 op/s 2023-12-14T18:33:11.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:11 smithi160 ceph-mon[43840]: osdmap e89: 12 total, 12 up, 12 in 2023-12-14T18:33:11.869 INFO:tasks.workunit.client.0.smithi093.stdout:ok 2023-12-14T18:33:11.874 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:33:11.874 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:33:11.934 INFO:tasks.workunit:Running workunit fs/misc/general_vxattrs.sh... 2023-12-14T18:33:11.934 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/general_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/general_vxattrs.sh 2023-12-14T18:33:11.966 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir -p dir 2023-12-14T18:33:11.969 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.pin dir 2023-12-14T18:33:11.969 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'ceph.dir.pin="-1"' 2023-12-14T18:33:11.971 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.dir.pin="-1" 2023-12-14T18:33:11.971 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin dir 2023-12-14T18:33:11.971 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:11.972 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:11.972 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin -v 1 dir 2023-12-14T18:33:11.974 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.pin dir 2023-12-14T18:33:11.974 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'ceph.dir.pin="1"' 2023-12-14T18:33:11.975 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.dir.pin="1" 2023-12-14T18:33:11.975 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.pin dir 2023-12-14T18:33:11.977 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.pin dir 2023-12-14T18:33:11.977 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'ceph.dir.pin="-1"' 2023-12-14T18:33:11.978 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.dir.pin="-1" 2023-12-14T18:33:11.979 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin.distributed dir 2023-12-14T18:33:11.979 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:11.979 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:11.980 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin.distributed -v 1 dir 2023-12-14T18:33:11.981 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.pin.distributed dir 2023-12-14T18:33:11.983 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin.random dir 2023-12-14T18:33:11.983 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:11.984 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:11.984 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.pin.random -v 0.01 dir 2023-12-14T18:33:11.985 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.pin.random dir 2023-12-14T18:33:11.987 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.quota.max_bytes dir 2023-12-14T18:33:11.988 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:11.988 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:11.988 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.quota.max_bytes -v 100000000 dir 2023-12-14T18:33:11.998 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.quota.max_bytes dir 2023-12-14T18:33:12.002 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.quota.max_files dir 2023-12-14T18:33:12.002 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:12.003 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:12.003 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.quota.max_files -v 10000 dir 2023-12-14T18:33:12.007 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.quota.max_files dir 2023-12-14T18:33:12.010 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir dir 2023-12-14T18:33:12.013 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo OK 2023-12-14T18:33:12.014 INFO:tasks.workunit.client.0.smithi093.stdout:OK 2023-12-14T18:33:12.014 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:33:12.014 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:33:12.091 INFO:tasks.workunit:Running workunit fs/misc/i_complete_vs_rename.sh... 2023-12-14T18:33:12.091 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/i_complete_vs_rename.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/i_complete_vs_rename.sh 2023-12-14T18:33:12.186 INFO:tasks.workunit.client.0.smithi093.stdout:a 2023-12-14T18:33:12.187 INFO:tasks.workunit.client.0.smithi093.stdout:b 2023-12-14T18:33:12.187 INFO:tasks.workunit.client.0.smithi093.stdout:c 2023-12-14T18:33:12.187 INFO:tasks.workunit.client.0.smithi093.stdout:d 2023-12-14T18:33:12.199 INFO:tasks.workunit.client.0.smithi093.stderr:stat: cannot statx 'e': No such file or directory 2023-12-14T18:33:12.203 INFO:tasks.workunit.client.0.smithi093.stdout:attempting rename over existing file... 2023-12-14T18:33:12.225 INFO:tasks.workunit.client.0.smithi093.stdout:f 2023-12-14T18:33:12.225 INFO:tasks.workunit.client.0.smithi093.stdout:rename over existing file is okay 2023-12-14T18:33:12.225 INFO:tasks.workunit.client.0.smithi093.stdout:attempting rename over negative dentry... 2023-12-14T18:33:12.234 INFO:tasks.workunit.client.0.smithi093.stdout:e 2023-12-14T18:33:12.234 INFO:tasks.workunit.client.0.smithi093.stdout:rename over negative dentry is ok 2023-12-14T18:33:12.234 INFO:tasks.workunit.client.0.smithi093.stdout:OK 2023-12-14T18:33:12.234 INFO:teuthology.orchestra.run:Running command with timeout 3600 2023-12-14T18:33:12.234 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2023-12-14T18:33:12.291 INFO:tasks.workunit:Running workunit fs/misc/layout_vxattrs.sh... 2023-12-14T18:33:12.291 DEBUG:teuthology.orchestra.run.smithi093:workunit test fs/misc/layout_vxattrs.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh 2023-12-14T18:33:12.314 INFO:tasks.workunit.client.0.smithi093.stderr:+ datapool= 2023-12-14T18:33:12.314 INFO:tasks.workunit.client.0.smithi093.stderr:+ dir=. 2023-12-14T18:33:12.314 INFO:tasks.workunit.client.0.smithi093.stderr:+ true 2023-12-14T18:33:12.314 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo . 2023-12-14T18:33:12.315 INFO:tasks.workunit.client.0.smithi093.stdout:. 2023-12-14T18:33:12.315 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfattr -n ceph.dir.layout.pool . --only-values 2023-12-14T18:33:12.315 INFO:tasks.workunit.client.0.smithi093.stderr:.: ceph.dir.layout.pool: No such attribute 2023-12-14T18:33:12.316 INFO:tasks.workunit.client.0.smithi093.stderr:+ datapool= 2023-12-14T18:33:12.316 INFO:tasks.workunit.client.0.smithi093.stderr:+ dir=./.. 2023-12-14T18:33:12.316 INFO:tasks.workunit.client.0.smithi093.stdout:./.. 2023-12-14T18:33:12.317 INFO:tasks.workunit.client.0.smithi093.stderr:+ true 2023-12-14T18:33:12.317 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo ./.. 2023-12-14T18:33:12.317 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfattr -n ceph.dir.layout.pool ./.. --only-values 2023-12-14T18:33:12.317 INFO:tasks.workunit.client.0.smithi093.stderr:./..: ceph.dir.layout.pool: No such attribute 2023-12-14T18:33:12.317 INFO:tasks.workunit.client.0.smithi093.stderr:+ datapool= 2023-12-14T18:33:12.318 INFO:tasks.workunit.client.0.smithi093.stderr:+ dir=./../.. 2023-12-14T18:33:12.318 INFO:tasks.workunit.client.0.smithi093.stderr:+ true 2023-12-14T18:33:12.318 INFO:tasks.workunit.client.0.smithi093.stdout:./../.. 2023-12-14T18:33:12.319 INFO:tasks.workunit.client.0.smithi093.stderr:+ echo ./../.. 2023-12-14T18:33:12.319 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfattr -n ceph.dir.layout.pool ./../.. --only-values 2023-12-14T18:33:12.319 INFO:tasks.workunit.client.0.smithi093.stderr:+ datapool=cephfs_data_ec 2023-12-14T18:33:12.319 INFO:tasks.workunit.client.0.smithi093.stderr:+ break 2023-12-14T18:33:12.319 INFO:tasks.workunit.client.0.smithi093.stderr:+ rm -f file file2 2023-12-14T18:33:12.319 INFO:tasks.workunit.client.0.smithi093.stderr:+ touch file file2 2023-12-14T18:33:12.321 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:33:12.322 INFO:tasks.workunit.client.0.smithi093.stdout:# file: file 2023-12-14T18:33:12.322 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.file.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2023-12-14T18:33:12.322 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-12-14T18:33:12.323 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:33:12.323 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q object_size= 2023-12-14T18:33:12.324 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:33:12.324 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q stripe_count= 2023-12-14T18:33:12.325 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:33:12.326 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q stripe_unit= 2023-12-14T18:33:12.327 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout file 2023-12-14T18:33:12.327 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q pool= 2023-12-14T18:33:12.327 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool file 2023-12-14T18:33:12.329 INFO:tasks.workunit.client.0.smithi093.stdout:# file: file 2023-12-14T18:33:12.329 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.file.layout.pool="cephfs_data_ec" 2023-12-14T18:33:12.329 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-12-14T18:33:12.329 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool_namespace file 2023-12-14T18:33:12.330 INFO:tasks.workunit.client.0.smithi093.stdout:# file: file 2023-12-14T18:33:12.330 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.file.layout.pool_namespace="" 2023-12-14T18:33:12.330 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-12-14T18:33:12.330 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_unit file 2023-12-14T18:33:12.331 INFO:tasks.workunit.client.0.smithi093.stdout:# file: file 2023-12-14T18:33:12.331 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.file.layout.stripe_unit="4194304" 2023-12-14T18:33:12.331 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-12-14T18:33:12.331 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_count file 2023-12-14T18:33:12.332 INFO:tasks.workunit.client.0.smithi093.stdout:# file: file 2023-12-14T18:33:12.332 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.file.layout.stripe_count="1" 2023-12-14T18:33:12.332 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-12-14T18:33:12.333 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.object_size file 2023-12-14T18:33:12.334 INFO:tasks.workunit.client.0.smithi093.stdout:# file: file 2023-12-14T18:33:12.334 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.file.layout.object_size="4194304" 2023-12-14T18:33:12.334 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-12-14T18:33:12.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.bogus file 2023-12-14T18:33:12.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 'No such attribute' 2023-12-14T18:33:12.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout file 2023-12-14T18:33:12.335 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 'No such attribute' 2023-12-14T18:33:12.336 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout.stripe_unit -v 1048576 file2 2023-12-14T18:33:12.342 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout.stripe_count -v 8 file2 2023-12-14T18:33:12.345 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout.object_size -v 10485760 file2 2023-12-14T18:33:12.350 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout.pool -v cephfs_data_ec file2 2023-12-14T18:33:12.356 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool file2 2023-12-14T18:33:12.356 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:33:12.357 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout.pool_namespace -v foons file2 2023-12-14T18:33:12.360 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:33:12.360 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q foons 2023-12-14T18:33:12.361 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.file.layout.pool_namespace file2 2023-12-14T18:33:12.365 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:33:12.365 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q -v foons 2023-12-14T18:33:12.367 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2023-12-14T18:33:12.367 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 1048576 2023-12-14T18:33:12.368 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:33:12.368 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 8 2023-12-14T18:33:12.369 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.object_size file2 2023-12-14T18:33:12.369 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 10485760 2023-12-14T18:33:12.371 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=4194304 stripe_count=16 object_size=41943040 pool=cephfs_data_ec pool_namespace=foons' file2 2023-12-14T18:33:12.375 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2023-12-14T18:33:12.375 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 4194304 2023-12-14T18:33:12.377 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:33:12.377 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 16 2023-12-14T18:33:12.379 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.object_size file2 2023-12-14T18:33:12.379 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 41943040 2023-12-14T18:33:12.380 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool file2 2023-12-14T18:33:12.380 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:33:12.382 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:33:12.382 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q foons 2023-12-14T18:33:12.384 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout -v stripe_unit=1048576 file2 2023-12-14T18:33:12.389 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2023-12-14T18:33:12.389 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 1048576 2023-12-14T18:33:12.390 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:33:12.391 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 16 2023-12-14T18:33:12.392 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.object_size file2 2023-12-14T18:33:12.392 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 41943040 2023-12-14T18:33:12.394 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool file2 2023-12-14T18:33:12.394 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:33:12.395 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:33:12.395 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q foons 2023-12-14T18:33:12.396 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout -v 'stripe_unit=2097152 stripe_count=4 object_size=2097152 pool=cephfs_data_ec pool_namespace=barns' file2 2023-12-14T18:33:12.400 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_unit file2 2023-12-14T18:33:12.400 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 2097152 2023-12-14T18:33:12.401 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:33:12.401 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 4 2023-12-14T18:33:12.402 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.object_size file2 2023-12-14T18:33:12.402 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 2097152 2023-12-14T18:33:12.403 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool file2 2023-12-14T18:33:12.403 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:33:12.404 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool_namespace file2 2023-12-14T18:33:12.404 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q barns 2023-12-14T18:33:12.406 INFO:tasks.workunit.client.0.smithi093.stderr:+ rm -f dir/file 2023-12-14T18:33:12.406 INFO:tasks.workunit.client.0.smithi093.stderr:+ rmdir dir 2023-12-14T18:33:12.407 INFO:tasks.workunit.client.0.smithi093.stderr:rmdir: failed to remove 'dir': No such file or directory 2023-12-14T18:33:12.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ true 2023-12-14T18:33:12.407 INFO:tasks.workunit.client.0.smithi093.stderr:+ mkdir -p dir 2023-12-14T18:33:12.409 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -d -m - dir 2023-12-14T18:33:12.409 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q ceph.dir.layout 2023-12-14T18:33:12.410 INFO:tasks.workunit.client.0.smithi093.stderr:+ true 2023-12-14T18:33:12.410 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -d -m - dir 2023-12-14T18:33:12.411 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q ceph.file.layout 2023-12-14T18:33:12.411 INFO:tasks.workunit.client.0.smithi093.stderr:+ true 2023-12-14T18:33:12.411 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:33:12.412 INFO:tasks.workunit.client.0.smithi093.stderr:dir: ceph.dir.layout: No such attribute 2023-12-14T18:33:12.412 INFO:tasks.workunit.client.0.smithi093.stderr:+ true 2023-12-14T18:33:12.412 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.layout.stripe_unit -v 1048576 dir 2023-12-14T18:33:12.416 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.layout.stripe_count -v 8 dir 2023-12-14T18:33:12.420 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.layout.object_size -v 10485760 dir 2023-12-14T18:33:12.423 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.layout.pool -v cephfs_data_ec dir 2023-12-14T18:33:12.426 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.layout.pool_namespace -v dirns dir 2023-12-14T18:33:12.430 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:33:12.431 INFO:tasks.workunit.client.0.smithi093.stdout:# file: dir 2023-12-14T18:33:12.431 INFO:tasks.workunit.client.0.smithi093.stdout:ceph.dir.layout="stripe_unit=1048576 stripe_count=8 object_size=10485760 pool=cephfs_data_ec pool_namespace=dirns" 2023-12-14T18:33:12.431 INFO:tasks.workunit.client.0.smithi093.stdout: 2023-12-14T18:33:12.431 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:33:12.431 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q object_size=10485760 2023-12-14T18:33:12.432 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:33:12.432 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q stripe_count=8 2023-12-14T18:33:12.434 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:33:12.434 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q stripe_unit=1048576 2023-12-14T18:33:12.435 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:33:12.435 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q pool=cephfs_data_ec 2023-12-14T18:33:12.436 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:33:12.436 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q pool_namespace=dirns 2023-12-14T18:33:12.438 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout.pool dir 2023-12-14T18:33:12.438 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:33:12.439 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout.stripe_unit dir 2023-12-14T18:33:12.439 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 1048576 2023-12-14T18:33:12.440 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout.stripe_count dir 2023-12-14T18:33:12.441 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 8 2023-12-14T18:33:12.442 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout.object_size dir 2023-12-14T18:33:12.442 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 10485760 2023-12-14T18:33:12.443 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout.pool_namespace dir 2023-12-14T18:33:12.443 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q dirns 2023-12-14T18:33:12.444 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout -v stripe_count=16 file2 2023-12-14T18:33:12.446 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:33:12.446 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 16 2023-12-14T18:33:12.447 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.file.layout -v 'object_size=10485760 stripe_count=8 stripe_unit=1048576 pool=cephfs_data_ec pool_namespace=dirns' file2 2023-12-14T18:33:12.451 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_count file2 2023-12-14T18:33:12.451 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 8 2023-12-14T18:33:12.452 INFO:tasks.workunit.client.0.smithi093.stderr:+ touch dir/file 2023-12-14T18:33:12.455 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool dir/file 2023-12-14T18:33:12.455 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q cephfs_data_ec 2023-12-14T18:33:12.456 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_unit dir/file 2023-12-14T18:33:12.456 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 1048576 2023-12-14T18:33:12.457 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.stripe_count dir/file 2023-12-14T18:33:12.457 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 8 2023-12-14T18:33:12.458 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.object_size dir/file 2023-12-14T18:33:12.458 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 10485760 2023-12-14T18:33:12.460 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.file.layout.pool_namespace dir/file 2023-12-14T18:33:12.460 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q dirns 2023-12-14T18:33:12.461 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.layout.pool dir 2023-12-14T18:33:12.461 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:12.466 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:12.466 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.layout.pool_id dir 2023-12-14T18:33:12.466 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:12.467 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:12.468 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.layout.pool_name dir 2023-12-14T18:33:12.468 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:12.469 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:12.470 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.layout.stripe_unit dir 2023-12-14T18:33:12.470 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:12.471 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:12.471 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.layout.stripe_count dir 2023-12-14T18:33:12.471 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:12.472 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:12.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.layout.object_size dir 2023-12-14T18:33:12.473 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir: Invalid argument' 2023-12-14T18:33:12.474 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir: Invalid argument 2023-12-14T18:33:12.475 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.file.layout.pool dir/file 2023-12-14T18:33:12.475 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:33:12.475 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:33:12.476 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.file.layout.pool_id dir/file 2023-12-14T18:33:12.476 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:33:12.477 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:33:12.477 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.file.layout.pool_name dir/file 2023-12-14T18:33:12.477 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:33:12.478 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:33:12.478 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.file.layout.stripe_unit dir/file 2023-12-14T18:33:12.478 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:33:12.479 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:33:12.479 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.file.layout.stripe_count dir/file 2023-12-14T18:33:12.479 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:33:12.480 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:33:12.480 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.file.layout.object_size dir/file 2023-12-14T18:33:12.480 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep 'setfattr: dir/file: Invalid argument' 2023-12-14T18:33:12.481 INFO:tasks.workunit.client.0.smithi093.stdout:setfattr: dir/file: Invalid argument 2023-12-14T18:33:12.481 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.file.layout.pool_namespace dir/file 2023-12-14T18:33:12.484 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.layout.pool_namespace dir 2023-12-14T18:33:12.489 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:33:12.489 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q -v pool_namespace=dirns 2023-12-14T18:33:12.490 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -x ceph.dir.layout dir 2023-12-14T18:33:12.494 INFO:tasks.workunit.client.0.smithi093.stderr:+ getfattr -n ceph.dir.layout dir 2023-12-14T18:33:12.494 INFO:tasks.workunit.client.0.smithi093.stderr:+ grep -q 'No such attribute' 2023-12-14T18:33:12.496 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfattr -n ceph.dir.layout . 2023-12-14T18:33:12.496 INFO:tasks.workunit.client.0.smithi093.stderr:++ sed -n 2,2p 2023-12-14T18:33:12.497 INFO:tasks.workunit.client.0.smithi093.stderr:.: ceph.dir.layout: No such attribute 2023-12-14T18:33:12.497 INFO:tasks.workunit.client.0.smithi093.stderr:+ DEFAULT_LAYOUT= 2023-12-14T18:33:12.497 INFO:tasks.workunit.client.0.smithi093.stderr:+ setfattr -n ceph.dir.layout -v 'stripe_unit=131072 stripe_count=2 object_size=131072' . 2023-12-14T18:33:12.501 INFO:tasks.workunit.client.0.smithi093.stderr:++ getfattr -n ceph.dir.layout . 2023-12-14T18:33:12.502 INFO:tasks.workunit.client.0.smithi093.stderr:++ sed -n 2,2p 2023-12-14T18:33:12.504 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:33:12.504 INFO:tasks.workunit.client.0.smithi093.stderr:+ CHANGED_LAYOUT='ceph.dir.layout="stripe_unit=131072 stripe_count=2 object_size=131072 pool=cephfs_data_ec"' 2023-12-14T18:33:12.504 INFO:tasks.workunit.client.0.smithi093.stderr:+ '[' 'ceph.dir.layout="stripe_unit=131072 stripe_count=2 object_size=131072 pool=cephfs_data_ec"' == 'ceph.dir.layout="stripe_unit=131072 stripe_count=2 object_size=131072 pool=cephfs.a.data"' ']' 2023-12-14T18:33:12.504 INFO:tasks.workunit:Stopping ['fs/misc'] on client.0... 2023-12-14T18:33:12.505 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2023-12-14T18:33:13.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:13 smithi093 ceph-mon[42070]: pgmap v207: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 139 KiB/s wr, 150 op/s 2023-12-14T18:33:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:13 smithi097 ceph-mon[43486]: pgmap v207: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 139 KiB/s wr, 150 op/s 2023-12-14T18:33:13.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:13 smithi160 ceph-mon[43840]: pgmap v207: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 81 KiB/s rd, 139 KiB/s wr, 150 op/s 2023-12-14T18:33:13.543 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/layout_vxattrs.sh) on smithi093 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh' 2023-12-14T18:33:13.655 ERROR:teuthology.util.sentry: Sentry event: https://sentry.ceph.com/organizations/ceph/?query=1e595971b5a149b2a9beaa6ee6faf175 Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 105, in run_tasks manager = run_one_task(taskname, ctx=ctx, config=config) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/run_tasks.py", line 83, in run_one_task return task(**kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 145, in task _spawn_on_all_clients(ctx, refspec, all_tasks, config.get('env'), File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 295, in _spawn_on_all_clients p.spawn(_run_tests, ctx, refspec, role, [unit], env, File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 84, in __exit__ for result in self: File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 98, in __next__ resurrect_traceback(result) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 30, in resurrect_traceback raise exc.exc_info[1] File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/parallel.py", line 23, in capture_traceback return func(*args, **kwargs) File "/home/teuthworker/src/github.com_chrisphoffman_ceph_bf026da9d69000f3d4a9d16360156095d6612a2d/qa/tasks/workunit.py", line 424, in _run_tests remote.run( File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/remote.py", line 523, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs) File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 455, in run r.wait() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 161, in wait self._raise_for_status() File "/home/teuthworker/src/git.ceph.com_teuthology_5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73/teuthology/orchestra/run.py", line 181, in _raise_for_status raise CommandFailedError( teuthology.exceptions.CommandFailedError: Command failed (workunit test fs/misc/layout_vxattrs.sh) on smithi093 with status 1: 'mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh' 2023-12-14T18:33:13.658 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2023-12-14T18:33:13.666 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:33:13.678 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2023-12-14T18:33:13.690 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2023-12-14T18:33:13.690 INFO:teuthology.task.exec:Executing custom commands... 2023-12-14T18:33:13.690 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:33:13.690 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2023-12-14T18:33:13.756 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2023-12-14T18:33:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:15 smithi093 ceph-mon[42070]: pgmap v208: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 KiB/s wr, 123 op/s 2023-12-14T18:33:15.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:15 smithi097 ceph-mon[43486]: pgmap v208: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 KiB/s wr, 123 op/s 2023-12-14T18:33:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:15 smithi160 ceph-mon[43840]: pgmap v208: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 17 KiB/s wr, 123 op/s 2023-12-14T18:33:17.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:17 smithi093 ceph-mon[42070]: pgmap v209: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 KiB/s wr, 119 op/s 2023-12-14T18:33:17.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:17 smithi097 ceph-mon[43486]: pgmap v209: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 KiB/s wr, 119 op/s 2023-12-14T18:33:17.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:17 smithi160 ceph-mon[43840]: pgmap v209: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 47 KiB/s wr, 119 op/s 2023-12-14T18:33:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:19 smithi093 ceph-mon[42070]: pgmap v210: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 106 op/s 2023-12-14T18:33:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:19 smithi097 ceph-mon[43486]: pgmap v210: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 106 op/s 2023-12-14T18:33:19.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:19 smithi160 ceph-mon[43840]: pgmap v210: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 41 KiB/s wr, 106 op/s 2023-12-14T18:33:21.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:21 smithi093 ceph-mon[42070]: pgmap v211: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-12-14T18:33:21.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:21 smithi097 ceph-mon[43486]: pgmap v211: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-12-14T18:33:21.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:21 smithi160 ceph-mon[43840]: pgmap v211: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 34 KiB/s wr, 11 op/s 2023-12-14T18:33:23.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:23 smithi093 ceph-mon[42070]: pgmap v212: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-12-14T18:33:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:23 smithi097 ceph-mon[43486]: pgmap v212: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-12-14T18:33:23.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:23 smithi160 ceph-mon[43840]: pgmap v212: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 31 KiB/s wr, 10 op/s 2023-12-14T18:33:25.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:25 smithi093 ceph-mon[42070]: pgmap v213: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-12-14T18:33:25.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:25 smithi097 ceph-mon[43486]: pgmap v213: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-12-14T18:33:25.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:25 smithi160 ceph-mon[43840]: pgmap v213: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 28 KiB/s wr, 9 op/s 2023-12-14T18:33:27.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:27 smithi093 ceph-mon[42070]: pgmap v214: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-12-14T18:33:27.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:27 smithi097 ceph-mon[43486]: pgmap v214: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-12-14T18:33:27.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:27 smithi160 ceph-mon[43840]: pgmap v214: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 32 KiB/s wr, 14 op/s 2023-12-14T18:33:29.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:29 smithi093 ceph-mon[42070]: pgmap v215: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 7 op/s 2023-12-14T18:33:29.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:29 smithi097 ceph-mon[43486]: pgmap v215: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 7 op/s 2023-12-14T18:33:29.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:29 smithi160 ceph-mon[43840]: pgmap v215: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 7 op/s 2023-12-14T18:33:31.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:31 smithi093 ceph-mon[42070]: pgmap v216: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2023-12-14T18:33:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:31 smithi097 ceph-mon[43486]: pgmap v216: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2023-12-14T18:33:31.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:31 smithi160 ceph-mon[43840]: pgmap v216: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.6 KiB/s wr, 10 op/s 2023-12-14T18:33:33.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:33 smithi093 ceph-mon[42070]: pgmap v217: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2023-12-14T18:33:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:33 smithi097 ceph-mon[43486]: pgmap v217: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2023-12-14T18:33:33.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:33 smithi160 ceph-mon[43840]: pgmap v217: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2023-12-14T18:33:35.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:35 smithi093 ceph-mon[42070]: pgmap v218: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2023-12-14T18:33:35.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:35 smithi097 ceph-mon[43486]: pgmap v218: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2023-12-14T18:33:35.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:35 smithi160 ceph-mon[43840]: pgmap v218: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 3.8 KiB/s wr, 7 op/s 2023-12-14T18:33:37.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:37 smithi093 ceph-mon[42070]: pgmap v219: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2023-12-14T18:33:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:37 smithi097 ceph-mon[43486]: pgmap v219: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2023-12-14T18:33:37.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:37 smithi160 ceph-mon[43840]: pgmap v219: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.9 KiB/s wr, 9 op/s 2023-12-14T18:33:39.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:39 smithi093 ceph-mon[42070]: pgmap v220: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:33:39.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:39 smithi097 ceph-mon[43486]: pgmap v220: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:33:39.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:39 smithi160 ceph-mon[43840]: pgmap v220: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:33:41.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:41 smithi093 ceph-mon[42070]: pgmap v221: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2023-12-14T18:33:41.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:41 smithi097 ceph-mon[43486]: pgmap v221: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2023-12-14T18:33:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:41 smithi160 ceph-mon[43840]: pgmap v221: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 3.1 KiB/s wr, 8 op/s 2023-12-14T18:33:43.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:43 smithi093 ceph-mon[42070]: pgmap v222: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:33:43.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:43 smithi097 ceph-mon[43486]: pgmap v222: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:33:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:43 smithi160 ceph-mon[43840]: pgmap v222: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:33:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:45 smithi093 ceph-mon[42070]: pgmap v223: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:33:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:45 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:33:45.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:45 smithi097 ceph-mon[43486]: pgmap v223: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:33:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:45 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:33:45.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:45 smithi160 ceph-mon[43840]: pgmap v223: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3.1 KiB/s wr, 5 op/s 2023-12-14T18:33:45.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:45 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:33:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:33:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:33:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:46 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:33:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:33:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:46 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:33:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:46 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:33:46.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:46 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:33:46.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:33:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:33:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:46 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:33:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:33:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:46 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:33:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:46 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:33:46.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:46 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:33:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:33:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:33:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:46 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:33:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:33:46.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:46 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:33:46.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:46 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:33:46.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:46 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:33:47.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:47 smithi093 ceph-mon[42070]: pgmap v224: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-12-14T18:33:47.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:47 smithi097 ceph-mon[43486]: pgmap v224: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-12-14T18:33:47.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:47 smithi160 ceph-mon[43840]: pgmap v224: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.3 KiB/s rd, 5.7 KiB/s wr, 8 op/s 2023-12-14T18:33:49.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:49 smithi093 ceph-mon[42070]: pgmap v225: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2023-12-14T18:33:49.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:49 smithi097 ceph-mon[43486]: pgmap v225: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2023-12-14T18:33:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:49 smithi160 ceph-mon[43840]: pgmap v225: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2023-12-14T18:33:51.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:51 smithi093 ceph-mon[42070]: pgmap v226: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-12-14T18:33:51.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:51 smithi097 ceph-mon[43486]: pgmap v226: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-12-14T18:33:51.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:51 smithi160 ceph-mon[43840]: pgmap v226: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 4.7 KiB/s wr, 8 op/s 2023-12-14T18:33:53.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:53 smithi093 ceph-mon[42070]: pgmap v227: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2023-12-14T18:33:53.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:53 smithi097 ceph-mon[43486]: pgmap v227: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2023-12-14T18:33:53.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:53 smithi160 ceph-mon[43840]: pgmap v227: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2023-12-14T18:33:55.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:55 smithi093 ceph-mon[42070]: pgmap v228: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2023-12-14T18:33:55.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:55 smithi097 ceph-mon[43486]: pgmap v228: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2023-12-14T18:33:55.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:55 smithi160 ceph-mon[43840]: pgmap v228: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 4.7 KiB/s wr, 5 op/s 2023-12-14T18:33:57.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:57 smithi093 ceph-mon[42070]: pgmap v229: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-12-14T18:33:57.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:57 smithi097 ceph-mon[43486]: pgmap v229: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-12-14T18:33:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:57 smithi160 ceph-mon[43840]: pgmap v229: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 6.2 KiB/s wr, 8 op/s 2023-12-14T18:33:59.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:33:59 smithi093 ceph-mon[42070]: pgmap v230: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2023-12-14T18:33:59.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:33:59 smithi097 ceph-mon[43486]: pgmap v230: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2023-12-14T18:33:59.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:33:59 smithi160 ceph-mon[43840]: pgmap v230: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 3.5 KiB/s wr, 5 op/s 2023-12-14T18:34:01.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:01 smithi093 ceph-mon[42070]: pgmap v231: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2023-12-14T18:34:01.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:01 smithi097 ceph-mon[43486]: pgmap v231: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2023-12-14T18:34:01.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:01 smithi160 ceph-mon[43840]: pgmap v231: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.5 KiB/s wr, 8 op/s 2023-12-14T18:34:03.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:03 smithi093 ceph-mon[42070]: pgmap v232: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:03.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:03 smithi097 ceph-mon[43486]: pgmap v232: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:03.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:03 smithi160 ceph-mon[43840]: pgmap v232: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:05.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:05 smithi093 ceph-mon[42070]: pgmap v233: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:05.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:05 smithi097 ceph-mon[43486]: pgmap v233: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:05.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:05 smithi160 ceph-mon[43840]: pgmap v233: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 1.4 KiB/s wr, 5 op/s 2023-12-14T18:34:07.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:07 smithi093 ceph-mon[42070]: pgmap v234: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 KiB/s wr, 22 op/s 2023-12-14T18:34:07.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:07 smithi093 ceph-mon[42070]: osdmap e90: 12 total, 12 up, 12 in 2023-12-14T18:34:07.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:07 smithi097 ceph-mon[43486]: pgmap v234: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 KiB/s wr, 22 op/s 2023-12-14T18:34:07.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:07 smithi097 ceph-mon[43486]: osdmap e90: 12 total, 12 up, 12 in 2023-12-14T18:34:07.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:07 smithi160 ceph-mon[43840]: pgmap v234: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 48 KiB/s wr, 22 op/s 2023-12-14T18:34:07.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:07 smithi160 ceph-mon[43840]: osdmap e90: 12 total, 12 up, 12 in 2023-12-14T18:34:09.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:09 smithi093 ceph-mon[42070]: pgmap v236: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-12-14T18:34:09.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:09 smithi097 ceph-mon[43486]: pgmap v236: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-12-14T18:34:09.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:09 smithi160 ceph-mon[43840]: pgmap v236: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 55 KiB/s wr, 23 op/s 2023-12-14T18:34:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:11 smithi093 ceph-mon[42070]: pgmap v237: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 55 KiB/s wr, 24 op/s 2023-12-14T18:34:11.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:11 smithi093 ceph-mon[42070]: osdmap e91: 12 total, 12 up, 12 in 2023-12-14T18:34:11.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:11 smithi097 ceph-mon[43486]: pgmap v237: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 55 KiB/s wr, 24 op/s 2023-12-14T18:34:11.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:11 smithi097 ceph-mon[43486]: osdmap e91: 12 total, 12 up, 12 in 2023-12-14T18:34:11.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:11 smithi160 ceph-mon[43840]: pgmap v237: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 55 KiB/s wr, 24 op/s 2023-12-14T18:34:11.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:11 smithi160 ceph-mon[43840]: osdmap e91: 12 total, 12 up, 12 in 2023-12-14T18:34:13.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:13 smithi093 ceph-mon[42070]: pgmap v239: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 30 op/s 2023-12-14T18:34:13.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:13 smithi097 ceph-mon[43486]: pgmap v239: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 30 op/s 2023-12-14T18:34:13.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:13 smithi160 ceph-mon[43840]: pgmap v239: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 19 KiB/s rd, 69 KiB/s wr, 30 op/s 2023-12-14T18:34:15.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:15 smithi093 ceph-mon[42070]: pgmap v240: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-12-14T18:34:15.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:15 smithi097 ceph-mon[43486]: pgmap v240: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-12-14T18:34:15.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:15 smithi160 ceph-mon[43840]: pgmap v240: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.0 KiB/s rd, 127 B/s wr, 3 op/s 2023-12-14T18:34:17.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:17 smithi093 ceph-mon[42070]: pgmap v241: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 227 B/s wr, 7 op/s 2023-12-14T18:34:17.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:17 smithi097 ceph-mon[43486]: pgmap v241: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 227 B/s wr, 7 op/s 2023-12-14T18:34:17.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:17 smithi160 ceph-mon[43840]: pgmap v241: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.6 KiB/s rd, 227 B/s wr, 7 op/s 2023-12-14T18:34:19.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:19 smithi093 ceph-mon[42070]: pgmap v242: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:34:19.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:19 smithi097 ceph-mon[43486]: pgmap v242: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:34:19.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:19 smithi160 ceph-mon[43840]: pgmap v242: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:34:21.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:21 smithi093 ceph-mon[42070]: pgmap v243: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2023-12-14T18:34:21.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:21 smithi097 ceph-mon[43486]: pgmap v243: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2023-12-14T18:34:21.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:21 smithi160 ceph-mon[43840]: pgmap v243: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 3.3 KiB/s wr, 7 op/s 2023-12-14T18:34:23.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:23 smithi093 ceph-mon[42070]: pgmap v244: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3 KiB/s wr, 6 op/s 2023-12-14T18:34:23.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:23 smithi097 ceph-mon[43486]: pgmap v244: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3 KiB/s wr, 6 op/s 2023-12-14T18:34:23.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:23 smithi160 ceph-mon[43840]: pgmap v244: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 3 KiB/s wr, 6 op/s 2023-12-14T18:34:25.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:25 smithi093 ceph-mon[42070]: pgmap v245: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2023-12-14T18:34:25.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:25 smithi097 ceph-mon[43486]: pgmap v245: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2023-12-14T18:34:25.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:25 smithi160 ceph-mon[43840]: pgmap v245: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2023-12-14T18:34:27.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:27 smithi093 ceph-mon[42070]: pgmap v246: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-12-14T18:34:27.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:27 smithi097 ceph-mon[43486]: pgmap v246: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-12-14T18:34:27.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:27 smithi160 ceph-mon[43840]: pgmap v246: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-12-14T18:34:29.815 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:29 smithi093 ceph-mon[42070]: pgmap v247: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2023-12-14T18:34:29.842 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:29 smithi097 ceph-mon[43486]: pgmap v247: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2023-12-14T18:34:29.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:29 smithi160 ceph-mon[43840]: pgmap v247: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2023-12-14T18:34:31.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:31 smithi097 ceph-mon[43486]: pgmap v248: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-12-14T18:34:31.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:31 smithi160 ceph-mon[43840]: pgmap v248: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-12-14T18:34:32.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:31 smithi093 ceph-mon[42070]: pgmap v248: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 5.1 KiB/s wr, 8 op/s 2023-12-14T18:34:33.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:33 smithi097 ceph-mon[43486]: pgmap v249: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-12-14T18:34:33.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:33 smithi160 ceph-mon[43840]: pgmap v249: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-12-14T18:34:34.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:33 smithi093 ceph-mon[42070]: pgmap v249: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-12-14T18:34:35.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:35 smithi097 ceph-mon[43486]: pgmap v250: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-12-14T18:34:35.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:35 smithi160 ceph-mon[43840]: pgmap v250: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-12-14T18:34:36.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:35 smithi093 ceph-mon[42070]: pgmap v250: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2023-12-14T18:34:37.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:37 smithi097 ceph-mon[43486]: pgmap v251: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2023-12-14T18:34:37.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:37 smithi160 ceph-mon[43840]: pgmap v251: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2023-12-14T18:34:38.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:37 smithi093 ceph-mon[42070]: pgmap v251: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 2.4 KiB/s wr, 7 op/s 2023-12-14T18:34:39.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:39 smithi097 ceph-mon[43486]: pgmap v252: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:34:39.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:39 smithi160 ceph-mon[43840]: pgmap v252: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:34:40.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:39 smithi093 ceph-mon[42070]: pgmap v252: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:34:41.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:41 smithi097 ceph-mon[43486]: pgmap v253: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-14T18:34:41.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:41 smithi160 ceph-mon[43840]: pgmap v253: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-14T18:34:42.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:41 smithi093 ceph-mon[42070]: pgmap v253: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-14T18:34:43.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:43 smithi097 ceph-mon[43486]: pgmap v254: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:43.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:43 smithi160 ceph-mon[43840]: pgmap v254: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:44.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:43 smithi093 ceph-mon[42070]: pgmap v254: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:45.843 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:45 smithi097 ceph-mon[43486]: pgmap v255: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:45.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:45 smithi160 ceph-mon[43840]: pgmap v255: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:46.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:45 smithi093 ceph-mon[42070]: pgmap v255: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:46.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:34:46.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:34:46.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:34:46.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:34:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:34:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:34:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:34:47.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:34:47.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:34:47.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:34:47.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:34:47.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:34:47.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:47 smithi160 ceph-mon[43840]: pgmap v256: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:47.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:47 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:34:47.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:47 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:34:47.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:47 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:34:47.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:47 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:34:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:47 smithi093 ceph-mon[42070]: pgmap v256: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:47 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:34:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:47 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:34:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:47 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:34:48.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:47 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:34:48.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:47 smithi097 ceph-mon[43486]: pgmap v256: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:48.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:47 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:34:48.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:47 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:34:48.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:47 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:34:48.094 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:47 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:34:49.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:49 smithi160 ceph-mon[43840]: pgmap v257: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:50.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:49 smithi093 ceph-mon[42070]: pgmap v257: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:50.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:49 smithi097 ceph-mon[43486]: pgmap v257: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:34:51.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:51 smithi160 ceph-mon[43840]: pgmap v258: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:52.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:51 smithi093 ceph-mon[42070]: pgmap v258: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:52.092 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:51 smithi097 ceph-mon[43486]: pgmap v258: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:53.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:53 smithi160 ceph-mon[43840]: pgmap v259: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-12-14T18:34:54.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:53 smithi093 ceph-mon[42070]: pgmap v259: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-12-14T18:34:54.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:53 smithi097 ceph-mon[43486]: pgmap v259: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-12-14T18:34:55.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:55 smithi160 ceph-mon[43840]: pgmap v260: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-12-14T18:34:56.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:55 smithi093 ceph-mon[42070]: pgmap v260: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-12-14T18:34:56.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:55 smithi097 ceph-mon[43486]: pgmap v260: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-12-14T18:34:57.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:57 smithi160 ceph-mon[43840]: pgmap v261: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:58.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:57 smithi093 ceph-mon[42070]: pgmap v261: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:58.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:57 smithi097 ceph-mon[43486]: pgmap v261: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:34:59.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:34:59 smithi160 ceph-mon[43840]: pgmap v262: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-12-14T18:35:00.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:34:59 smithi093 ceph-mon[42070]: pgmap v262: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-12-14T18:35:00.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:34:59 smithi097 ceph-mon[43486]: pgmap v262: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.4 KiB/s rd, 4 op/s 2023-12-14T18:35:01.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:01 smithi160 ceph-mon[43840]: pgmap v263: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:02.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:01 smithi093 ceph-mon[42070]: pgmap v263: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:02.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:01 smithi097 ceph-mon[43486]: pgmap v263: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:03.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:03 smithi160 ceph-mon[43840]: pgmap v264: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:04.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:03 smithi093 ceph-mon[42070]: pgmap v264: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:04.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:03 smithi097 ceph-mon[43486]: pgmap v264: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:05.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:05 smithi160 ceph-mon[43840]: pgmap v265: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:06.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:05 smithi093 ceph-mon[42070]: pgmap v265: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:06.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:05 smithi097 ceph-mon[43486]: pgmap v265: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:07.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:07 smithi160 ceph-mon[43840]: pgmap v266: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:35:07.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:07 smithi160 ceph-mon[43840]: osdmap e92: 12 total, 12 up, 12 in 2023-12-14T18:35:08.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:07 smithi093 ceph-mon[42070]: pgmap v266: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:35:08.068 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:07 smithi093 ceph-mon[42070]: osdmap e92: 12 total, 12 up, 12 in 2023-12-14T18:35:08.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:07 smithi097 ceph-mon[43486]: pgmap v266: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:35:08.095 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:07 smithi097 ceph-mon[43486]: osdmap e92: 12 total, 12 up, 12 in 2023-12-14T18:35:09.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:09 smithi160 ceph-mon[43840]: pgmap v268: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:10.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:09 smithi093 ceph-mon[42070]: pgmap v268: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:10.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:09 smithi097 ceph-mon[43486]: pgmap v268: 129 pgs: 129 active+clean; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:11.997 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:11 smithi160 ceph-mon[43840]: pgmap v269: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:12.066 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:11 smithi093 ceph-mon[42070]: pgmap v269: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:12.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:11 smithi097 ceph-mon[43486]: pgmap v269: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:13.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:12 smithi160 ceph-mon[43840]: pgmap v270: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:13.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:12 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:13.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:12 smithi093 ceph-mon[42070]: pgmap v270: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:13.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:12 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:12 smithi097 ceph-mon[43486]: pgmap v270: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:13.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:12 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:15.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:14 smithi160 ceph-mon[43840]: pgmap v271: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:15.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:14 smithi093 ceph-mon[42070]: pgmap v271: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:15.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:14 smithi097 ceph-mon[43486]: pgmap v271: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 732 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:35:17.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:16 smithi160 ceph-mon[43840]: pgmap v272: 129 pgs: 8 active+clean+snaptrim, 100 active+clean, 21 active+clean+snaptrim_wait; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:17.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:16 smithi093 ceph-mon[42070]: pgmap v272: 129 pgs: 8 active+clean+snaptrim, 100 active+clean, 21 active+clean+snaptrim_wait; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:17.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:16 smithi097 ceph-mon[43486]: pgmap v272: 129 pgs: 8 active+clean+snaptrim, 100 active+clean, 21 active+clean+snaptrim_wait; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.2 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:35:18.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:17 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:18.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:17 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:18.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:17 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:18 smithi160 ceph-mon[43840]: pgmap v273: 129 pgs: 8 active+clean+snaptrim, 100 active+clean, 21 active+clean+snaptrim_wait; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 186 B/s wr, 5 op/s 2023-12-14T18:35:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:18 smithi093 ceph-mon[42070]: pgmap v273: 129 pgs: 8 active+clean+snaptrim, 100 active+clean, 21 active+clean+snaptrim_wait; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 186 B/s wr, 5 op/s 2023-12-14T18:35:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:18 smithi097 ceph-mon[43486]: pgmap v273: 129 pgs: 8 active+clean+snaptrim, 100 active+clean, 21 active+clean+snaptrim_wait; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.9 KiB/s rd, 186 B/s wr, 5 op/s 2023-12-14T18:35:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:20 smithi160 ceph-mon[43840]: pgmap v274: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-12-14T18:35:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:20 smithi160 ceph-mon[43840]: osdmap e93: 12 total, 12 up, 12 in 2023-12-14T18:35:21.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:20 smithi093 ceph-mon[42070]: pgmap v274: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-12-14T18:35:21.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:20 smithi093 ceph-mon[42070]: osdmap e93: 12 total, 12 up, 12 in 2023-12-14T18:35:21.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:20 smithi097 ceph-mon[43486]: pgmap v274: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 170 B/s wr, 7 op/s 2023-12-14T18:35:21.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:20 smithi097 ceph-mon[43486]: osdmap e93: 12 total, 12 up, 12 in 2023-12-14T18:35:23.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:22 smithi160 ceph-mon[43840]: pgmap v276: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:23.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:22 smithi093 ceph-mon[42070]: pgmap v276: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:23.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:22 smithi097 ceph-mon[43486]: pgmap v276: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:24 smithi160 ceph-mon[43840]: pgmap v277: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:25.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:24 smithi093 ceph-mon[42070]: pgmap v277: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:25.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:24 smithi097 ceph-mon[43486]: pgmap v277: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 102 B/s wr, 6 op/s 2023-12-14T18:35:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:26 smithi160 ceph-mon[43840]: pgmap v278: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:27.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:26 smithi093 ceph-mon[42070]: pgmap v278: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:27.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:26 smithi097 ceph-mon[43486]: pgmap v278: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:28 smithi160 ceph-mon[43840]: pgmap v279: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:29.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:28 smithi093 ceph-mon[42070]: pgmap v279: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:29.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:28 smithi097 ceph-mon[43486]: pgmap v279: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:31.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:30 smithi160 ceph-mon[43840]: pgmap v280: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:31.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:30 smithi093 ceph-mon[42070]: pgmap v280: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:31.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:30 smithi097 ceph-mon[43486]: pgmap v280: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:35:33.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:32 smithi160 ceph-mon[43840]: pgmap v281: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-12-14T18:35:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:32 smithi093 ceph-mon[42070]: pgmap v281: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-12-14T18:35:33.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:32 smithi097 ceph-mon[43486]: pgmap v281: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-12-14T18:35:35.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:34 smithi160 ceph-mon[43840]: pgmap v282: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:35.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:34 smithi093 ceph-mon[42070]: pgmap v282: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:35.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:34 smithi097 ceph-mon[43486]: pgmap v282: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:36 smithi160 ceph-mon[43840]: pgmap v283: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:37.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:36 smithi093 ceph-mon[42070]: pgmap v283: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:36 smithi097 ceph-mon[43486]: pgmap v283: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:39.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:38 smithi160 ceph-mon[43840]: pgmap v284: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:39.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:38 smithi093 ceph-mon[42070]: pgmap v284: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:39.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:38 smithi097 ceph-mon[43486]: pgmap v284: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:41.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:40 smithi160 ceph-mon[43840]: pgmap v285: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:41.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:40 smithi093 ceph-mon[42070]: pgmap v285: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:41.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:40 smithi097 ceph-mon[43486]: pgmap v285: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:43.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:42 smithi160 ceph-mon[43840]: pgmap v286: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:43.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:42 smithi093 ceph-mon[42070]: pgmap v286: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:43.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:42 smithi097 ceph-mon[43486]: pgmap v286: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:45.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:44 smithi160 ceph-mon[43840]: pgmap v287: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:45.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:44 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:44 smithi093 ceph-mon[42070]: pgmap v287: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:44 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:45.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:44 smithi097 ceph-mon[43486]: pgmap v287: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:45.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:44 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:46 smithi160 ceph-mon[43840]: pgmap v288: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:47.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:35:47.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:35:47.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:35:47.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:46 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:35:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:46 smithi093 ceph-mon[42070]: pgmap v288: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:35:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:35:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:35:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:46 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:35:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:46 smithi097 ceph-mon[43486]: pgmap v288: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:35:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:35:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:35:47.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:46 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:35:48.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:47 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:35:48.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:47 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:35:48.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:47 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:35:48.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:47 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:35:48.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:47 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:48.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:47 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:35:48.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:47 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:35:48.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:47 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:35:48.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:47 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:35:48.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:47 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:48.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:47 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:35:48.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:47 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:35:48.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:47 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:35:48.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:47 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:35:48.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:47 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:35:49.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:48 smithi160 ceph-mon[43840]: pgmap v289: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:49.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:48 smithi093 ceph-mon[42070]: pgmap v289: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:49.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:48 smithi097 ceph-mon[43486]: pgmap v289: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:50 smithi160 ceph-mon[43840]: pgmap v290: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:51.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:50 smithi093 ceph-mon[42070]: pgmap v290: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:51.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:50 smithi097 ceph-mon[43486]: pgmap v290: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:53.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:52 smithi160 ceph-mon[43840]: pgmap v291: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:53.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:52 smithi093 ceph-mon[42070]: pgmap v291: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:53.348 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:52 smithi097 ceph-mon[43486]: pgmap v291: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:55.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:54 smithi160 ceph-mon[43840]: pgmap v292: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:55.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:54 smithi093 ceph-mon[42070]: pgmap v292: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:55.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:54 smithi097 ceph-mon[43486]: pgmap v292: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:57.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:56 smithi160 ceph-mon[43840]: pgmap v293: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:57.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:56 smithi093 ceph-mon[42070]: pgmap v293: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:57.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:56 smithi097 ceph-mon[43486]: pgmap v293: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:35:59.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:58 smithi160 ceph-mon[43840]: pgmap v294: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:59.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:58 smithi093 ceph-mon[42070]: pgmap v294: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:35:59.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:58 smithi097 ceph-mon[43486]: pgmap v294: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:00.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:35:59 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:00.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:35:59 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:00.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:35:59 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:01.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:01 smithi093 ceph-mon[42070]: pgmap v295: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:01.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:01 smithi097 ceph-mon[43486]: pgmap v295: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:01.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:01 smithi160 ceph-mon[43840]: pgmap v295: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:03.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:03 smithi093 ceph-mon[42070]: pgmap v296: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:03.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:03 smithi093 ceph-mon[42070]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:03.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:03 smithi097 ceph-mon[43486]: pgmap v296: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:03.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:03 smithi097 ceph-mon[43486]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:03.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:03 smithi160 ceph-mon[43840]: pgmap v296: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:03.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:03 smithi160 ceph-mon[43840]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=e=up:active} 3 up:standby-replay 6 up:standby 2023-12-14T18:36:05.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:05 smithi093 ceph-mon[42070]: pgmap v297: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:05.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:05 smithi097 ceph-mon[43486]: pgmap v297: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:05.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:05 smithi160 ceph-mon[43840]: pgmap v297: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:07.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:06 smithi160 ceph-mon[43840]: pgmap v298: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:36:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:06 smithi093 ceph-mon[42070]: pgmap v298: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:36:07.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:06 smithi097 ceph-mon[43486]: pgmap v298: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 22 op/s 2023-12-14T18:36:08.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:07 smithi160 ceph-mon[43840]: osdmap e94: 12 total, 12 up, 12 in 2023-12-14T18:36:08.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:07 smithi093 ceph-mon[42070]: osdmap e94: 12 total, 12 up, 12 in 2023-12-14T18:36:08.347 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:07 smithi097 ceph-mon[43486]: osdmap e94: 12 total, 12 up, 12 in 2023-12-14T18:36:09.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:09 smithi093 ceph-mon[42070]: pgmap v300: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:09.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:09 smithi097 ceph-mon[43486]: pgmap v300: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:09.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:09 smithi160 ceph-mon[43840]: pgmap v300: 129 pgs: 129 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:11.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:11 smithi093 ceph-mon[42070]: pgmap v301: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:11.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:11 smithi097 ceph-mon[43486]: pgmap v301: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:11.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:11 smithi160 ceph-mon[43840]: pgmap v301: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:13.317 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:13 smithi093 ceph-mon[42070]: pgmap v302: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:13.344 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:13 smithi097 ceph-mon[43486]: pgmap v302: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:13.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:13 smithi160 ceph-mon[43840]: pgmap v302: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:15.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:15 smithi097 ceph-mon[43486]: pgmap v303: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:15.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:15 smithi160 ceph-mon[43840]: pgmap v303: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:15.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:15 smithi093 ceph-mon[42070]: pgmap v303: 129 pgs: 24 active+clean+snaptrim, 72 active+clean+snaptrim_wait, 33 active+clean; 732 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 23 op/s 2023-12-14T18:36:17.250 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:16 smithi160 ceph-mon[43840]: pgmap v304: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 732 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:36:17.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:16 smithi093 ceph-mon[42070]: pgmap v304: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 732 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:36:17.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:16 smithi097 ceph-mon[43486]: pgmap v304: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 732 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 KiB/s rd, 204 B/s wr, 6 op/s 2023-12-14T18:36:19.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:18 smithi160 ceph-mon[43840]: pgmap v305: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 732 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 186 B/s wr, 5 op/s 2023-12-14T18:36:19.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:18 smithi093 ceph-mon[42070]: pgmap v305: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 732 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 186 B/s wr, 5 op/s 2023-12-14T18:36:19.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:18 smithi097 ceph-mon[43486]: pgmap v305: 129 pgs: 23 active+clean+snaptrim, 70 active+clean+snaptrim_wait, 36 active+clean; 732 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 2.8 KiB/s rd, 186 B/s wr, 5 op/s 2023-12-14T18:36:21.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:20 smithi160 ceph-mon[43840]: pgmap v306: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2023-12-14T18:36:21.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:20 smithi093 ceph-mon[42070]: pgmap v306: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2023-12-14T18:36:21.346 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:20 smithi097 ceph-mon[43486]: pgmap v306: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.8 KiB/s rd, 170 B/s wr, 7 op/s 2023-12-14T18:36:23.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:22 smithi160 ceph-mon[43840]: pgmap v307: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:36:23.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:22 smithi093 ceph-mon[42070]: pgmap v307: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:36:23.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:22 smithi097 ceph-mon[43486]: pgmap v307: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:36:25.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:24 smithi160 ceph-mon[43840]: pgmap v308: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:36:25.316 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:24 smithi093 ceph-mon[42070]: pgmap v308: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:36:25.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:24 smithi097 ceph-mon[43486]: pgmap v308: 129 pgs: 21 active+clean+snaptrim, 66 active+clean+snaptrim_wait, 42 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 85 B/s wr, 5 op/s 2023-12-14T18:36:27.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:26 smithi160 ceph-mon[43840]: pgmap v309: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-14T18:36:27.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:26 smithi093 ceph-mon[42070]: pgmap v309: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-14T18:36:27.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:26 smithi097 ceph-mon[43486]: pgmap v309: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 85 B/s wr, 7 op/s 2023-12-14T18:36:29.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:28 smithi160 ceph-mon[43840]: pgmap v310: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:29.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:28 smithi093 ceph-mon[42070]: pgmap v310: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:29.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:28 smithi097 ceph-mon[43486]: pgmap v310: 129 pgs: 15 active+clean+snaptrim, 50 active+clean+snaptrim_wait, 64 active+clean; 732 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:30.874 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2023-12-14T18:36:31.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.027+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/1851962686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb41048d0 msgr2=0x7f0cb4104ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.027+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/1851962686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb41048d0 0x7f0cb4104ca0 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0cb0009990 tx=0x7f0cb0009ca0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.027+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/1851962686 shutdown_connections 2023-12-14T18:36:31.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.027+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/1851962686 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb41048d0 0x7f0cb4104ca0 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.027+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/1851962686 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb40ff270 0x7f0cb410b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.027+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/1851962686 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40fe8c0 0x7f0cb40fed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.027+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/1851962686 >> 172.21.15.93:0/1851962686 conn(0x7f0cb4075810 msgr2=0x7f0cb4075c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:31.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.027+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/1851962686 shutdown_connections 2023-12-14T18:36:31.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.027+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/1851962686 wait complete. 2023-12-14T18:36:31.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cbc1a4700 1 Processor -- start 2023-12-14T18:36:31.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cbc1a4700 1 -- start start 2023-12-14T18:36:31.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cbc1a4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40fe8c0 0x7f0cb41941c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cbc1a4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb40ff270 0x7f0cb4194700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cbc1a4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb41048d0 0x7f0cb4198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cbc1a4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4199150 con 0x7f0cb40ff270 2023-12-14T18:36:31.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cbc1a4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb41992c0 con 0x7f0cb40fe8c0 2023-12-14T18:36:31.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cbc1a4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cb4199480 con 0x7f0cb41048d0 2023-12-14T18:36:31.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cb973f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb40ff270 0x7f0cb4194700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cb973f700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb40ff270 0x7f0cb4194700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:36644/0 (socket says 172.21.15.93:36644) 2023-12-14T18:36:31.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cb973f700 1 -- 172.21.15.93:0/3729214034 learned_addr learned my addr 172.21.15.93:0/3729214034 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:31.032 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cb9f40700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40fe8c0 0x7f0cb41941c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.028+0000 7f0cba741700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb41048d0 0x7f0cb4198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0cb973f700 1 -- 172.21.15.93:0/3729214034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb41048d0 msgr2=0x7f0cb4198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0cb973f700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb41048d0 0x7f0cb4198b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0cb973f700 1 -- 172.21.15.93:0/3729214034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40fe8c0 msgr2=0x7f0cb41941c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0cb973f700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40fe8c0 0x7f0cb41941c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0cb973f700 1 -- 172.21.15.93:0/3729214034 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb00095f0 con 0x7f0cb40ff270 2023-12-14T18:36:31.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0cb973f700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb40ff270 0x7f0cb4194700 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400fe70 tx=0x7f0ca400d6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:31.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0caaffd700 1 -- 172.21.15.93:0/3729214034 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca400a8a0 con 0x7f0cb40ff270 2023-12-14T18:36:31.033 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cb410e0c0 con 0x7f0cb40ff270 2023-12-14T18:36:31.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0caaffd700 1 -- 172.21.15.93:0/3729214034 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f0ca400de30 con 0x7f0cb40ff270 2023-12-14T18:36:31.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0caaffd700 1 -- 172.21.15.93:0/3729214034 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ca400cb40 con 0x7f0cb40ff270 2023-12-14T18:36:31.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.029+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cb410df30 con 0x7f0cb40ff270 2023-12-14T18:36:31.034 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.030+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cb410e250 con 0x7f0cb40ff270 2023-12-14T18:36:31.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.031+0000 7f0caaffd700 1 -- 172.21.15.93:0/3729214034 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f0ca400cca0 con 0x7f0cb40ff270 2023-12-14T18:36:31.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.032+0000 7f0caaffd700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f0ca00a12c0 0x7f0ca00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.032+0000 7f0cb9f40700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f0ca00a12c0 0x7f0ca00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.032+0000 7f0caaffd700 1 -- 172.21.15.93:0/3729214034 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f0ca40ce870 con 0x7f0cb40ff270 2023-12-14T18:36:31.035 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.033+0000 7f0cb9f40700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f0ca00a12c0 0x7f0ca00a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000b2e0 tx=0x7f0cb0005fd0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:31.037 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.036+0000 7f0caaffd700 1 -- 172.21.15.93:0/3729214034 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f0ca40978d0 con 0x7f0cb40ff270 2023-12-14T18:36:31.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:30 smithi160 ceph-mon[43840]: pgmap v311: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:31.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:30 smithi160 ceph-mon[43840]: osdmap e95: 12 total, 12 up, 12 in 2023-12-14T18:36:31.255 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.255+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 --> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) v1 -- 0x7f0cb40fd5e0 con 0x7f0ca00a12c0 2023-12-14T18:36:31.268 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.267+0000 7f0caaffd700 1 -- 172.21.15.93:0/3729214034 <== mgr.25100 v2:172.21.15.93:6800/3274300600 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+335 (secure 0 0 0) 0x7f0cb40fd5e0 con 0x7f0ca00a12c0 2023-12-14T18:36:31.268 INFO:teuthology.orchestra.run.smithi093.stdout:{"fs": "cephfs", "subvol": null, "path": "/", "rel_path": "/", "schedule": "1M", "retention": {"h": 3, "M": 6}, "start": "2023-12-14T00:00:00", "created": "2023-12-14T18:26:28", "first": "2023-12-14T18:27:00", "last": "2023-12-14T18:36:00", "last_pruned": "2023-12-14T18:36:00", "created_count": 10, "pruned_count": 4, "active": true} 2023-12-14T18:36:31.270 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.269+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f0ca00a12c0 msgr2=0x7f0ca00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.269+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f0ca00a12c0 0x7f0ca00a3770 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f0cb000b2e0 tx=0x7f0cb0005fd0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.269+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb40ff270 msgr2=0x7f0cb4194700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.269+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb40ff270 0x7f0cb4194700 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0ca400fe70 tx=0x7f0ca400d6c0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.270+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 shutdown_connections 2023-12-14T18:36:31.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.270+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f0cb41048d0 0x7f0cb4198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.270+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f0cb40fe8c0 0x7f0cb41941c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.271 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.270+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f0ca00a12c0 0x7f0ca00a3770 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.270+0000 7f0cbc1a4700 1 --2- 172.21.15.93:0/3729214034 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f0cb40ff270 0x7f0cb4194700 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.270+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 >> 172.21.15.93:0/3729214034 conn(0x7f0cb4075810 msgr2=0x7f0cb40fced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:31.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.270+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 shutdown_connections 2023-12-14T18:36:31.272 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.270+0000 7f0cbc1a4700 1 -- 172.21.15.93:0/3729214034 wait complete. 2023-12-14T18:36:31.281 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:30 smithi093 ceph-mon[42070]: pgmap v311: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:31.281 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:30 smithi093 ceph-mon[42070]: osdmap e95: 12 total, 12 up, 12 in 2023-12-14T18:36:31.282 DEBUG:teuthology.orchestra.run.smithi093:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2023-12-14T18:36:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:30 smithi097 ceph-mon[43486]: pgmap v311: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:31.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:30 smithi097 ceph-mon[43486]: osdmap e95: 12 total, 12 up, 12 in 2023-12-14T18:36:31.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 -- 172.21.15.93:0/3391854735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ffd50 msgr2=0x7f4974103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 --2- 172.21.15.93:0/3391854735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ffd50 0x7f4974103d10 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f49700099c0 tx=0x7f4970009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 -- 172.21.15.93:0/3391854735 shutdown_connections 2023-12-14T18:36:31.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 --2- 172.21.15.93:0/3391854735 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49741043f0 0x7f497410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 --2- 172.21.15.93:0/3391854735 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49740ff3b0 0x7f49740ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 --2- 172.21.15.93:0/3391854735 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ffd50 0x7f4974103d10 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 -- 172.21.15.93:0/3391854735 >> 172.21.15.93:0/3391854735 conn(0x7f4974075f70 msgr2=0x7f4974076370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:31.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 -- 172.21.15.93:0/3391854735 shutdown_connections 2023-12-14T18:36:31.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 -- 172.21.15.93:0/3391854735 wait complete. 2023-12-14T18:36:31.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 Processor -- start 2023-12-14T18:36:31.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.405+0000 7f497bd2e700 1 -- start start 2023-12-14T18:36:31.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f497bd2e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ff3b0 0x7f4974195250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f497bd2e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49740ffd50 0x7f4974195790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f497bd2e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49741043f0 0x7f4974199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f497bd2e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497419a230 con 0x7f49740ff3b0 2023-12-14T18:36:31.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f497bd2e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497419a3a0 con 0x7f49741043f0 2023-12-14T18:36:31.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f497bd2e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497419a560 con 0x7f49740ffd50 2023-12-14T18:36:31.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f4979aca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ff3b0 0x7f4974195250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f4979aca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ff3b0 0x7f4974195250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44624/0 (socket says 172.21.15.93:44624) 2023-12-14T18:36:31.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f4979aca700 1 -- 172.21.15.93:0/740435435 learned_addr learned my addr 172.21.15.93:0/740435435 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:31.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f497a2cb700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49741043f0 0x7f4974199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f49792c9700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49740ffd50 0x7f4974195790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f4979aca700 1 -- 172.21.15.93:0/740435435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49740ffd50 msgr2=0x7f4974195790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f4979aca700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49740ffd50 0x7f4974195790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f4979aca700 1 -- 172.21.15.93:0/740435435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49741043f0 msgr2=0x7f4974199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f4979aca700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49741043f0 0x7f4974199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f4979aca700 1 -- 172.21.15.93:0/740435435 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49700095f0 con 0x7f49740ff3b0 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f49792c9700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49740ffd50 0x7f4974195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f4979aca700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ff3b0 0x7f4974195250 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f496400d980 tx=0x7f496400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f496affd700 1 -- 172.21.15.93:0/740435435 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964004220 con 0x7f49740ff3b0 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f496affd700 1 -- 172.21.15.93:0/740435435 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4964007640 con 0x7f49740ff3b0 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f497410f1c0 con 0x7f49740ff3b0 2023-12-14T18:36:31.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f496affd700 1 -- 172.21.15.93:0/740435435 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4964005610 con 0x7f49740ff3b0 2023-12-14T18:36:31.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.406+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f497410f030 con 0x7f49740ff3b0 2023-12-14T18:36:31.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.407+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497410f350 con 0x7f49740ff3b0 2023-12-14T18:36:31.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.408+0000 7f496affd700 1 -- 172.21.15.93:0/740435435 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4964004750 con 0x7f49740ff3b0 2023-12-14T18:36:31.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.409+0000 7f496affd700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f49600a1340 0x7f49600a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.409+0000 7f496affd700 1 -- 172.21.15.93:0/740435435 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f49640cd350 con 0x7f49740ff3b0 2023-12-14T18:36:31.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.409+0000 7f49792c9700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f49600a1340 0x7f49600a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.409+0000 7f49792c9700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f49600a1340 0x7f49600a37f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f497000b580 tx=0x7f4970005fb0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:31.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.412+0000 7f496affd700 1 -- 172.21.15.93:0/740435435 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4964096410 con 0x7f49740ff3b0 2023-12-14T18:36:31.630 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.629+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 --> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f49740fd370 con 0x7f49600a1340 2023-12-14T18:36:31.638 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.637+0000 7f496affd700 1 -- 172.21.15.93:0/740435435 <== mgr.25100 v2:172.21.15.93:6800/3274300600 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+10 (secure 0 0 0) 0x7f49740fd370 con 0x7f49600a1340 2023-12-14T18:36:31.638 INFO:teuthology.orchestra.run.smithi093.stdout:/ 1M 6M3h 2023-12-14T18:36:31.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f49600a1340 msgr2=0x7f49600a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f49600a1340 0x7f49600a37f0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f497000b580 tx=0x7f4970005fb0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ff3b0 msgr2=0x7f4974195250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ff3b0 0x7f4974195250 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f496400d980 tx=0x7f496400dc90 comp rx=0 tx=0).stop 2023-12-14T18:36:31.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 shutdown_connections 2023-12-14T18:36:31.641 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f49600a1340 0x7f49600a37f0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f49740ff3b0 0x7f4974195250 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f49741043f0 0x7f4974199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 --2- 172.21.15.93:0/740435435 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f49740ffd50 0x7f4974195790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 >> 172.21.15.93:0/740435435 conn(0x7f4974075f70 msgr2=0x7f49740fcc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:31.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 shutdown_connections 2023-12-14T18:36:31.642 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.640+0000 7f497bd2e700 1 -- 172.21.15.93:0/740435435 wait complete. 2023-12-14T18:36:31.653 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2023-12-14T18:36:31.666 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:36:31.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.787+0000 7f9d92fe4700 1 -- 172.21.15.93:0/3814701442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 msgr2=0x7f9d8c100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.787+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/3814701442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 0x7f9d8c100e50 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f9d78009990 tx=0x7f9d78009ca0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.787+0000 7f9d92fe4700 1 -- 172.21.15.93:0/3814701442 shutdown_connections 2023-12-14T18:36:31.790 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.787+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/3814701442 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d8c101390 0x7f9d8c10d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.787+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/3814701442 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d8c1051b0 0x7f9d8c105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.787+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/3814701442 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 0x7f9d8c100e50 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.787+0000 7f9d92fe4700 1 -- 172.21.15.93:0/3814701442 >> 172.21.15.93:0/3814701442 conn(0x7f9d8c077520 msgr2=0x7f9d8c077920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:31.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.787+0000 7f9d92fe4700 1 -- 172.21.15.93:0/3814701442 shutdown_connections 2023-12-14T18:36:31.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.787+0000 7f9d92fe4700 1 -- 172.21.15.93:0/3814701442 wait complete. 2023-12-14T18:36:31.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d92fe4700 1 Processor -- start 2023-12-14T18:36:31.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d92fe4700 1 -- start start 2023-12-14T18:36:31.791 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d92fe4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 0x7f9d8c196320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d92fe4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d8c101390 0x7f9d8c196860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d92fe4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d8c1051b0 0x7f9d8c19acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d92fe4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c19b300 con 0x7f9d8c1009e0 2023-12-14T18:36:31.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d92fe4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c19b470 con 0x7f9d8c101390 2023-12-14T18:36:31.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d92fe4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8c19b630 con 0x7f9d8c1051b0 2023-12-14T18:36:31.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d90d80700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 0x7f9d8c196320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d90d80700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 0x7f9d8c196320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44630/0 (socket says 172.21.15.93:44630) 2023-12-14T18:36:31.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d90d80700 1 -- 172.21.15.93:0/1870523988 learned_addr learned my addr 172.21.15.93:0/1870523988 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:31.792 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d8bfff700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d8c101390 0x7f9d8c196860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d91581700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d8c1051b0 0x7f9d8c19acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d90d80700 1 -- 172.21.15.93:0/1870523988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d8c1051b0 msgr2=0x7f9d8c19acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d90d80700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d8c1051b0 0x7f9d8c19acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d90d80700 1 -- 172.21.15.93:0/1870523988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d8c101390 msgr2=0x7f9d8c196860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d90d80700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d8c101390 0x7f9d8c196860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d90d80700 1 -- 172.21.15.93:0/1870523988 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d780095f0 con 0x7f9d8c1009e0 2023-12-14T18:36:31.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.788+0000 7f9d91581700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d8c1051b0 0x7f9d8c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:31.793 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.789+0000 7f9d90d80700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 0x7f9d8c196320 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000d980 tx=0x7f9d8000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:31.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.789+0000 7f9d89ffb700 1 -- 172.21.15.93:0/1870523988 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80004220 con 0x7f9d8c1009e0 2023-12-14T18:36:31.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.789+0000 7f9d89ffb700 1 -- 172.21.15.93:0/1870523988 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9d80007640 con 0x7f9d8c1009e0 2023-12-14T18:36:31.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.789+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d8c110320 con 0x7f9d8c1009e0 2023-12-14T18:36:31.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.789+0000 7f9d89ffb700 1 -- 172.21.15.93:0/1870523988 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d80005610 con 0x7f9d8c1009e0 2023-12-14T18:36:31.794 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.789+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8c110190 con 0x7f9d8c1009e0 2023-12-14T18:36:31.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.790+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d8c1104b0 con 0x7f9d8c1009e0 2023-12-14T18:36:31.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.791+0000 7f9d89ffb700 1 -- 172.21.15.93:0/1870523988 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9d80004750 con 0x7f9d8c1009e0 2023-12-14T18:36:31.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.791+0000 7f9d89ffb700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9d7c0a12c0 0x7f9d7c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:31.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.792+0000 7f9d89ffb700 1 -- 172.21.15.93:0/1870523988 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9d800ccf00 con 0x7f9d8c1009e0 2023-12-14T18:36:31.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.792+0000 7f9d8bfff700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9d7c0a12c0 0x7f9d7c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:31.795 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.792+0000 7f9d8bfff700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9d7c0a12c0 0x7f9d7c0a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800b2e0 tx=0x7f9d78005960 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:31.796 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.796+0000 7f9d89ffb700 1 -- 172.21.15.93:0/1870523988 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9d80095fc0 con 0x7f9d8c1009e0 2023-12-14T18:36:31.990 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.989+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9d8c067160 con 0x7f9d8c1009e0 2023-12-14T18:36:31.992 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.991+0000 7f9d89ffb700 1 -- 172.21.15.93:0/1870523988 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+24913 (secure 0 0 0) 0x7f9d80016020 con 0x7f9d8c1009e0 2023-12-14T18:36:31.992 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:36:31.993 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":95,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:36:30.939319+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1584909638":"2023-12-15T18:26:27.742320+0000","172.21.15.93:0/1420703523":"2023-12-15T18:26:27.742320+0000","172.21.15.93:0/932543927":"2023-12-15T18:26:27.742320+0000","172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6801/3802499407":"2023-12-15T18:26:27.742320+0000","172.21.15.93:6800/3802499407":"2023-12-15T18:26:27.742320+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/978480379":"2023-12-15T18:26:27.742320+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:36:31.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9d7c0a12c0 msgr2=0x7f9d7c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.994 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9d7c0a12c0 0x7f9d7c0a3770 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f9d7800b2e0 tx=0x7f9d78005960 comp rx=0 tx=0).stop 2023-12-14T18:36:31.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 msgr2=0x7f9d8c196320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:31.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 0x7f9d8c196320 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000d980 tx=0x7f9d8000dc90 comp rx=0 tx=0).stop 2023-12-14T18:36:31.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 shutdown_connections 2023-12-14T18:36:31.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9d7c0a12c0 0x7f9d7c0a3770 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9d8c1009e0 0x7f9d8c196320 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9d8c1051b0 0x7f9d8c19acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 --2- 172.21.15.93:0/1870523988 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9d8c101390 0x7f9d8c196860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:31.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 >> 172.21.15.93:0/1870523988 conn(0x7f9d8c077520 msgr2=0x7f9d8c0ff6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:31.995 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 shutdown_connections 2023-12-14T18:36:31.996 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:31.993+0000 7f9d92fe4700 1 -- 172.21.15.93:0/1870523988 wait complete. 2023-12-14T18:36:32.007 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:24:34.407045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:24:35.728548+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:24:38.796189+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:36:32.007 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:36:32.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 -- 172.21.15.93:0/1396587341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ffd70 msgr2=0x7f7644103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 --2- 172.21.15.93:0/1396587341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ffd70 0x7f7644103d30 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f76340099c0 tx=0x7f7634009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 -- 172.21.15.93:0/1396587341 shutdown_connections 2023-12-14T18:36:32.132 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 --2- 172.21.15.93:0/1396587341 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7644104410 0x7f764410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 --2- 172.21.15.93:0/1396587341 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76440ff3d0 0x7f76440ff7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 --2- 172.21.15.93:0/1396587341 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ffd70 0x7f7644103d30 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 -- 172.21.15.93:0/1396587341 >> 172.21.15.93:0/1396587341 conn(0x7f7644076000 msgr2=0x7f7644076400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:32.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 -- 172.21.15.93:0/1396587341 shutdown_connections 2023-12-14T18:36:32.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 -- 172.21.15.93:0/1396587341 wait complete. 2023-12-14T18:36:32.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 Processor -- start 2023-12-14T18:36:32.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.131+0000 7f764a97d700 1 -- start start 2023-12-14T18:36:32.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f764a97d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ff3d0 0x7f76441971d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f764a97d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76440ffd70 0x7f7644197710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f764a97d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7644104410 0x7f76441a2250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f764a97d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7644198050 con 0x7f76440ff3d0 2023-12-14T18:36:32.135 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f764a97d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76441a2790 con 0x7f7644104410 2023-12-14T18:36:32.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f764a97d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76441a2950 con 0x7f76440ffd70 2023-12-14T18:36:32.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7643fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ff3d0 0x7f76441971d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7643fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ff3d0 0x7f76441971d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44648/0 (socket says 172.21.15.93:44648) 2023-12-14T18:36:32.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7643fff700 1 -- 172.21.15.93:0/867245511 learned_addr learned my addr 172.21.15.93:0/867245511 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:32.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7648f1a700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7644104410 0x7f76441a2250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7643fff700 1 -- 172.21.15.93:0/867245511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76440ffd70 msgr2=0x7f7644197710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7643fff700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76440ffd70 0x7f7644197710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7643fff700 1 -- 172.21.15.93:0/867245511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7644104410 msgr2=0x7f76441a2250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.136 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7643fff700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7644104410 0x7f76441a2250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7643fff700 1 -- 172.21.15.93:0/867245511 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76340095f0 con 0x7f76440ff3d0 2023-12-14T18:36:32.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f7643fff700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ff3d0 0x7f76441971d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f763800c920 tx=0x7f763800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:32.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f76417fa700 1 -- 172.21.15.93:0/867245511 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638007b00 con 0x7f76440ff3d0 2023-12-14T18:36:32.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f764410f190 con 0x7f76440ff3d0 2023-12-14T18:36:32.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f76417fa700 1 -- 172.21.15.93:0/867245511 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f763800ce70 con 0x7f76440ff3d0 2023-12-14T18:36:32.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f76417fa700 1 -- 172.21.15.93:0/867245511 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7638005460 con 0x7f76440ff3d0 2023-12-14T18:36:32.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.132+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f764410f000 con 0x7f76440ff3d0 2023-12-14T18:36:32.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.133+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f764410f320 con 0x7f76440ff3d0 2023-12-14T18:36:32.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.135+0000 7f76417fa700 1 -- 172.21.15.93:0/867245511 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f76380055c0 con 0x7f76440ff3d0 2023-12-14T18:36:32.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.136+0000 7f76417fa700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f762c0a12c0 0x7f762c0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.136+0000 7f76437fe700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f762c0a12c0 0x7f762c0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.138 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.136+0000 7f76417fa700 1 -- 172.21.15.93:0/867245511 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f76380cd0e0 con 0x7f76440ff3d0 2023-12-14T18:36:32.139 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.136+0000 7f76437fe700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f762c0a12c0 0x7f762c0a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7634006010 tx=0x7f7634000bc0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:32.140 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.139+0000 7f76417fa700 1 -- 172.21.15.93:0/867245511 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7638096140 con 0x7f76440ff3d0 2023-12-14T18:36:32.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:31 smithi160 ceph-mon[43840]: from='client.16014 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:36:32.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:31 smithi160 ceph-mon[43840]: from='client.16020 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:36:32.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:31 smithi093 ceph-mon[42070]: from='client.16014 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:36:32.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:31 smithi093 ceph-mon[42070]: from='client.16020 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:36:32.329 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.328+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76441a3460 con 0x7f76440ff3d0 2023-12-14T18:36:32.330 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.329+0000 7f76417fa700 1 -- 172.21.15.93:0/867245511 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v95) v1 ==== 93+0+10 (secure 0 0 0) 0x7f763801d020 con 0x7f76440ff3d0 2023-12-14T18:36:32.330 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:36:32.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f762c0a12c0 msgr2=0x7f762c0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f762c0a12c0 0x7f762c0a3770 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7634006010 tx=0x7f7634000bc0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.332 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ff3d0 msgr2=0x7f76441971d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ff3d0 0x7f76441971d0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f763800c920 tx=0x7f763800cc30 comp rx=0 tx=0).stop 2023-12-14T18:36:32.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 shutdown_connections 2023-12-14T18:36:32.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f762c0a12c0 0x7f762c0a3770 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f76440ff3d0 0x7f76441971d0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7644104410 0x7f76441a2250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 --2- 172.21.15.93:0/867245511 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f76440ffd70 0x7f7644197710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 >> 172.21.15.93:0/867245511 conn(0x7f7644076000 msgr2=0x7f76440fcc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:32.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 shutdown_connections 2023-12-14T18:36:32.333 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.331+0000 7f764a97d700 1 -- 172.21.15.93:0/867245511 wait complete. 2023-12-14T18:36:32.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:31 smithi097 ceph-mon[43486]: from='client.16014 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:36:32.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:31 smithi097 ceph-mon[43486]: from='client.16020 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2023-12-14T18:36:32.344 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:36:32.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.467+0000 7fa240020700 1 -- 172.21.15.93:0/1448623433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 msgr2=0x7fa238068e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.467+0000 7fa240020700 1 --2- 172.21.15.93:0/1448623433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 0x7fa238068e80 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fa234009990 tx=0x7fa234009ca0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.467+0000 7fa240020700 1 -- 172.21.15.93:0/1448623433 shutdown_connections 2023-12-14T18:36:32.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.467+0000 7fa240020700 1 --2- 172.21.15.93:0/1448623433 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 0x7fa238068e80 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.467+0000 7fa240020700 1 --2- 172.21.15.93:0/1448623433 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa238102c00 0x7fa2381030a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.467+0000 7fa240020700 1 --2- 172.21.15.93:0/1448623433 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa2380693c0 0x7fa238069830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.471 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.467+0000 7fa240020700 1 -- 172.21.15.93:0/1448623433 >> 172.21.15.93:0/1448623433 conn(0x7fa238075fa0 msgr2=0x7fa2380763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:32.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.467+0000 7fa240020700 1 -- 172.21.15.93:0/1448623433 shutdown_connections 2023-12-14T18:36:32.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa240020700 1 -- 172.21.15.93:0/1448623433 wait complete. 2023-12-14T18:36:32.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa240020700 1 Processor -- start 2023-12-14T18:36:32.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa240020700 1 -- start start 2023-12-14T18:36:32.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa240020700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2380693c0 0x7fa2381a2700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa240020700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa238102c00 0x7fa2381a2c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa240020700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 0x7fa23819c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa240020700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23819ce30 con 0x7fa238105100 2023-12-14T18:36:32.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa240020700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23819cfa0 con 0x7fa2380693c0 2023-12-14T18:36:32.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa240020700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa23819d160 con 0x7fa238102c00 2023-12-14T18:36:32.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa23e5bd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 0x7fa23819c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa23e5bd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 0x7fa23819c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44666/0 (socket says 172.21.15.93:44666) 2023-12-14T18:36:32.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa23e5bd700 1 -- 172.21.15.93:0/2834741739 learned_addr learned my addr 172.21.15.93:0/2834741739 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:32.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.468+0000 7fa23ddbc700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2380693c0 0x7fa2381a2700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa23d5bb700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa238102c00 0x7fa2381a2c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa23e5bd700 1 -- 172.21.15.93:0/2834741739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa238102c00 msgr2=0x7fa2381a2c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa23e5bd700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa238102c00 0x7fa2381a2c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.473 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa23e5bd700 1 -- 172.21.15.93:0/2834741739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2380693c0 msgr2=0x7fa2381a2700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa23e5bd700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2380693c0 0x7fa2381a2700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa23e5bd700 1 -- 172.21.15.93:0/2834741739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2340095f0 con 0x7fa238105100 2023-12-14T18:36:32.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa23e5bd700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 0x7fa23819c7d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa230007f10 tx=0x7fa23000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:32.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa22effd700 1 -- 172.21.15.93:0/2834741739 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230009ea0 con 0x7fa238105100 2023-12-14T18:36:32.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa22effd700 1 -- 172.21.15.93:0/2834741739 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fa23000f040 con 0x7fa238105100 2023-12-14T18:36:32.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa238116860 con 0x7fa238105100 2023-12-14T18:36:32.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa22effd700 1 -- 172.21.15.93:0/2834741739 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa230004810 con 0x7fa238105100 2023-12-14T18:36:32.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.469+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2381166d0 con 0x7fa238105100 2023-12-14T18:36:32.474 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.470+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2381169f0 con 0x7fa238105100 2023-12-14T18:36:32.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.471+0000 7fa22effd700 1 -- 172.21.15.93:0/2834741739 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fa230003a00 con 0x7fa238105100 2023-12-14T18:36:32.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.471+0000 7fa22effd700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fa2240a12c0 0x7fa2240a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.475 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.472+0000 7fa22effd700 1 -- 172.21.15.93:0/2834741739 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fa2300cc8e0 con 0x7fa238105100 2023-12-14T18:36:32.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.472+0000 7fa23ddbc700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fa2240a12c0 0x7fa2240a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.476 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.472+0000 7fa23ddbc700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fa2240a12c0 0x7fa2240a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa23400b6f0 tx=0x7fa234005fd0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:32.477 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.476+0000 7fa22effd700 1 -- 172.21.15.93:0/2834741739 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fa23004e730 con 0x7fa238105100 2023-12-14T18:36:32.666 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.666+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa2381a9a70 con 0x7fa238105100 2023-12-14T18:36:32.667 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.666+0000 7fa22effd700 1 -- 172.21.15.93:0/2834741739 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v95) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa230095900 con 0x7fa238105100 2023-12-14T18:36:32.667 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:36:32.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fa2240a12c0 msgr2=0x7fa2240a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.669 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fa2240a12c0 0x7fa2240a3770 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fa23400b6f0 tx=0x7fa234005fd0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 msgr2=0x7fa23819c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 0x7fa23819c7d0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fa230007f10 tx=0x7fa23000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 shutdown_connections 2023-12-14T18:36:32.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fa2240a12c0 0x7fa2240a3770 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fa238105100 0x7fa23819c7d0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fa2380693c0 0x7fa2381a2700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 --2- 172.21.15.93:0/2834741739 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fa238102c00 0x7fa2381a2c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 >> 172.21.15.93:0/2834741739 conn(0x7fa238075fa0 msgr2=0x7fa238101be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:32.670 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 shutdown_connections 2023-12-14T18:36:32.671 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.668+0000 7fa240020700 1 -- 172.21.15.93:0/2834741739 wait complete. 2023-12-14T18:36:32.681 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:36:32.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 -- 172.21.15.93:0/3048535188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 msgr2=0x7fe5400ff780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 --2- 172.21.15.93:0/3048535188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 0x7fe5400ff780 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fe5340099c0 tx=0x7fe534009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 -- 172.21.15.93:0/3048535188 shutdown_connections 2023-12-14T18:36:32.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 --2- 172.21.15.93:0/3048535188 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 0x7fe5400ff780 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 --2- 172.21.15.93:0/3048535188 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5401043f0 0x7fe54010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.809 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 --2- 172.21.15.93:0/3048535188 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5400ffd50 0x7fe540103d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 -- 172.21.15.93:0/3048535188 >> 172.21.15.93:0/3048535188 conn(0x7fe540075fa0 msgr2=0x7fe5400763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 -- 172.21.15.93:0/3048535188 shutdown_connections 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 -- 172.21.15.93:0/3048535188 wait complete. 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 Processor -- start 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.806+0000 7fe546f52700 1 -- start start 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe546f52700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 0x7fe540100db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe546f52700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5400ffd50 0x7fe5401012f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe546f52700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5401043f0 0x7fe5401a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe546f52700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe540101b20 con 0x7fe5400ff3b0 2023-12-14T18:36:32.810 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe546f52700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5401a4af0 con 0x7fe5401043f0 2023-12-14T18:36:32.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe546f52700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5401a4c30 con 0x7fe5400ffd50 2023-12-14T18:36:32.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe544cee700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 0x7fe540100db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe544cee700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 0x7fe540100db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44682/0 (socket says 172.21.15.93:44682) 2023-12-14T18:36:32.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe544cee700 1 -- 172.21.15.93:0/3521825190 learned_addr learned my addr 172.21.15.93:0/3521825190 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:32.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe53ffff700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5400ffd50 0x7fe5401012f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe5454ef700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5401043f0 0x7fe5401a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe544cee700 1 -- 172.21.15.93:0/3521825190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5400ffd50 msgr2=0x7fe5401012f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe544cee700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5400ffd50 0x7fe5401012f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.811 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe544cee700 1 -- 172.21.15.93:0/3521825190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5401043f0 msgr2=0x7fe5401a45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:32.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe544cee700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5401043f0 0x7fe5401a45b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:32.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe544cee700 1 -- 172.21.15.93:0/3521825190 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5340095f0 con 0x7fe5400ff3b0 2023-12-14T18:36:32.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe5454ef700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5401043f0 0x7fe5401a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:32.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe544cee700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 0x7fe540100db0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe534004730 tx=0x7fe534004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:32.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe53dffb700 1 -- 172.21.15.93:0/3521825190 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe53401d070 con 0x7fe5400ff3b0 2023-12-14T18:36:32.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe53dffb700 1 -- 172.21.15.93:0/3521825190 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fe534004c90 con 0x7fe5400ff3b0 2023-12-14T18:36:32.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe54010f200 con 0x7fe5400ff3b0 2023-12-14T18:36:32.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.807+0000 7fe53dffb700 1 -- 172.21.15.93:0/3521825190 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe534005720 con 0x7fe5400ff3b0 2023-12-14T18:36:32.812 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.808+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe54010f070 con 0x7fe5400ff3b0 2023-12-14T18:36:32.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.808+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe54010f390 con 0x7fe5400ff3b0 2023-12-14T18:36:32.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.810+0000 7fe53dffb700 1 -- 172.21.15.93:0/3521825190 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fe534005000 con 0x7fe5400ff3b0 2023-12-14T18:36:32.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.810+0000 7fe53dffb700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fe5300a12c0 0x7fe5300a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:32.813 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.811+0000 7fe53ffff700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fe5300a12c0 0x7fe5300a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:32.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.811+0000 7fe53dffb700 1 -- 172.21.15.93:0/3521825190 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9366+0+0 (secure 0 0 0) 0x7fe5340cde90 con 0x7fe5400ff3b0 2023-12-14T18:36:32.814 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.811+0000 7fe53ffff700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fe5300a12c0 0x7fe5300a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe5401025a0 tx=0x7fe52c00b410 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:32.815 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:32.814+0000 7fe53dffb700 1 -- 172.21.15.93:0/3521825190 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fe534096fe0 con 0x7fe5400ff3b0 2023-12-14T18:36:33.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.006+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe5401a54e0 con 0x7fe5400ff3b0 2023-12-14T18:36:33.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.006+0000 7fe53dffb700 1 -- 172.21.15.93:0/3521825190 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe5340203b0 con 0x7fe5400ff3b0 2023-12-14T18:36:33.007 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-12-14T18:36:33.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.008+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fe5300a12c0 msgr2=0x7fe5300a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.008+0000 7fe546f52700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fe5300a12c0 0x7fe5300a3770 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe5401025a0 tx=0x7fe52c00b410 comp rx=0 tx=0).stop 2023-12-14T18:36:33.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.008+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 msgr2=0x7fe540100db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.008+0000 7fe546f52700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 0x7fe540100db0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fe534004730 tx=0x7fe534004760 comp rx=0 tx=0).stop 2023-12-14T18:36:33.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.008+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 shutdown_connections 2023-12-14T18:36:33.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.009+0000 7fe546f52700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fe5300a12c0 0x7fe5300a3770 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.009+0000 7fe546f52700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fe5400ff3b0 0x7fe540100db0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.009+0000 7fe546f52700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fe5401043f0 0x7fe5401a45b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.009+0000 7fe546f52700 1 --2- 172.21.15.93:0/3521825190 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fe5400ffd50 0x7fe5401012f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.009+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 >> 172.21.15.93:0/3521825190 conn(0x7fe540075fa0 msgr2=0x7fe540068e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:33.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.009+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 shutdown_connections 2023-12-14T18:36:33.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.009+0000 7fe546f52700 1 -- 172.21.15.93:0/3521825190 wait complete. 2023-12-14T18:36:33.022 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:36:33.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.145+0000 7f9c08181700 1 -- 172.21.15.93:0/892660490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ffd50 msgr2=0x7f9c00103d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.145+0000 7f9c08181700 1 --2- 172.21.15.93:0/892660490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ffd50 0x7f9c00103d10 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0099c0 tx=0x7f9bfc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.145+0000 7f9c08181700 1 -- 172.21.15.93:0/892660490 shutdown_connections 2023-12-14T18:36:33.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.145+0000 7f9c08181700 1 --2- 172.21.15.93:0/892660490 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c001043f0 0x7f9c0010bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.145+0000 7f9c08181700 1 --2- 172.21.15.93:0/892660490 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c000ff3b0 0x7f9c000ff780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.145+0000 7f9c08181700 1 --2- 172.21.15.93:0/892660490 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ffd50 0x7f9c00103d10 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.145+0000 7f9c08181700 1 -- 172.21.15.93:0/892660490 >> 172.21.15.93:0/892660490 conn(0x7f9c00075fa0 msgr2=0x7f9c000763a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:33.147 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.145+0000 7f9c08181700 1 -- 172.21.15.93:0/892660490 shutdown_connections 2023-12-14T18:36:33.148 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.145+0000 7f9c08181700 1 -- 172.21.15.93:0/892660490 wait complete. 2023-12-14T18:36:33.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.146+0000 7f9c08181700 1 Processor -- start 2023-12-14T18:36:33.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.146+0000 7f9c08181700 1 -- start start 2023-12-14T18:36:33.150 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c08181700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ff3b0 0x7f9c001952a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:33.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c08181700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c000ffd50 0x7f9c001957e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:33.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c08181700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c001043f0 0x7f9c00199c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:33.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c08181700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0019a280 con 0x7f9c000ff3b0 2023-12-14T18:36:33.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c08181700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0019a3f0 con 0x7f9c000ffd50 2023-12-14T18:36:33.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c08181700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c0019a5b0 con 0x7f9c001043f0 2023-12-14T18:36:33.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c05f1d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ff3b0 0x7f9c001952a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:33.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c05f1d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ff3b0 0x7f9c001952a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44704/0 (socket says 172.21.15.93:44704) 2023-12-14T18:36:33.151 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c05f1d700 1 -- 172.21.15.93:0/16513905 learned_addr learned my addr 172.21.15.93:0/16513905 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:33.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c0571c700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c000ffd50 0x7f9c001957e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:33.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c0671e700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c001043f0 0x7f9c00199c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:33.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c05f1d700 1 -- 172.21.15.93:0/16513905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c001043f0 msgr2=0x7f9c00199c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c05f1d700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c001043f0 0x7f9c00199c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c05f1d700 1 -- 172.21.15.93:0/16513905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c000ffd50 msgr2=0x7f9c001957e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c05f1d700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c000ffd50 0x7f9c001957e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c05f1d700 1 -- 172.21.15.93:0/16513905 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bfc0095f0 con 0x7f9c000ff3b0 2023-12-14T18:36:33.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c0671e700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c001043f0 0x7f9c00199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:33.152 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c0571c700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c000ffd50 0x7f9c001957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:33.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c05f1d700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ff3b0 0x7f9c001952a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000eab0 tx=0x7f9bf000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:33.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9bf6ffd700 1 -- 172.21.15.93:0/16513905 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0013070 con 0x7f9c000ff3b0 2023-12-14T18:36:33.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9bf6ffd700 1 -- 172.21.15.93:0/16513905 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f9bf0017410 con 0x7f9c000ff3b0 2023-12-14T18:36:33.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c0010f170 con 0x7f9c000ff3b0 2023-12-14T18:36:33.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9bf6ffd700 1 -- 172.21.15.93:0/16513905 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bf0016610 con 0x7f9c000ff3b0 2023-12-14T18:36:33.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.147+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c0010efe0 con 0x7f9c000ff3b0 2023-12-14T18:36:33.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.148+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c0010f300 con 0x7f9c000ff3b0 2023-12-14T18:36:33.153 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.149+0000 7f9bf6ffd700 1 -- 172.21.15.93:0/16513905 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f9bf0017880 con 0x7f9c000ff3b0 2023-12-14T18:36:33.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.150+0000 7f9bf6ffd700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9bec0a12c0 0x7f9bec0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:33.154 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.150+0000 7f9bf6ffd700 1 -- 172.21.15.93:0/16513905 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f9bf00cebe0 con 0x7f9c000ff3b0 2023-12-14T18:36:33.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.150+0000 7f9c0571c700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9bec0a12c0 0x7f9bec0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:33.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.150+0000 7f9c0571c700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9bec0a12c0 0x7f9bec0a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0058e0 tx=0x7f9bfc01a040 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:33.155 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.153+0000 7f9bf6ffd700 1 -- 172.21.15.93:0/16513905 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f9bf0097ca0 con 0x7f9c000ff3b0 2023-12-14T18:36:33.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:32 smithi160 ceph-mon[43840]: pgmap v313: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:32 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1870523988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:36:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:32 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/867245511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:36:33.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:32 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2834741739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:36:33.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:32 smithi093 ceph-mon[42070]: pgmap v313: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:32 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1870523988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:36:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:32 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/867245511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:36:33.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:32 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2834741739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:36:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:32 smithi097 ceph-mon[43486]: pgmap v313: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:32 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1870523988' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:36:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:32 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/867245511' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:36:33.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:32 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2834741739' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:36:33.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.348+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f9c001965f0 con 0x7f9c000ff3b0 2023-12-14T18:36:33.349 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.348+0000 7f9bf6ffd700 1 -- 172.21.15.93:0/16513905 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v95) v1 ==== 103+0+11 (secure 0 0 0) 0x7f9bf009b2c0 con 0x7f9c000ff3b0 2023-12-14T18:36:33.349 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:36:33.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9bec0a12c0 msgr2=0x7f9bec0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9bec0a12c0 0x7f9bec0a3770 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f9bfc0058e0 tx=0x7f9bfc01a040 comp rx=0 tx=0).stop 2023-12-14T18:36:33.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ff3b0 msgr2=0x7f9c001952a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ff3b0 0x7f9c001952a0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f9bf000eab0 tx=0x7f9bf000ee70 comp rx=0 tx=0).stop 2023-12-14T18:36:33.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 shutdown_connections 2023-12-14T18:36:33.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f9bec0a12c0 0x7f9bec0a3770 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f9c000ff3b0 0x7f9c001952a0 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f9c001043f0 0x7f9c00199c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 --2- 172.21.15.93:0/16513905 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f9c000ffd50 0x7f9c001957e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 >> 172.21.15.93:0/16513905 conn(0x7f9c00075fa0 msgr2=0x7f9c000fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:33.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 shutdown_connections 2023-12-14T18:36:33.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.350+0000 7f9c08181700 1 -- 172.21.15.93:0/16513905 wait complete. 2023-12-14T18:36:33.363 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2023-12-14T18:36:33.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.484+0000 7f56befca700 1 -- 172.21.15.93:0/2458087829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81009e0 msgr2=0x7f56b8100e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.488 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.484+0000 7f56befca700 1 --2- 172.21.15.93:0/2458087829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81009e0 0x7f56b8100e50 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f56a4009990 tx=0x7f56a4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.484+0000 7f56befca700 1 -- 172.21.15.93:0/2458087829 shutdown_connections 2023-12-14T18:36:33.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.484+0000 7f56befca700 1 --2- 172.21.15.93:0/2458087829 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56b8101390 0x7f56b810d060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.484+0000 7f56befca700 1 --2- 172.21.15.93:0/2458087829 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b81051b0 0x7f56b8105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.484+0000 7f56befca700 1 --2- 172.21.15.93:0/2458087829 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81009e0 0x7f56b8100e50 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.484+0000 7f56befca700 1 -- 172.21.15.93:0/2458087829 >> 172.21.15.93:0/2458087829 conn(0x7f56b8077520 msgr2=0x7f56b8077920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:33.489 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.484+0000 7f56befca700 1 -- 172.21.15.93:0/2458087829 shutdown_connections 2023-12-14T18:36:33.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.484+0000 7f56befca700 1 -- 172.21.15.93:0/2458087829 wait complete. 2023-12-14T18:36:33.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56befca700 1 Processor -- start 2023-12-14T18:36:33.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56befca700 1 -- start start 2023-12-14T18:36:33.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56befca700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56b81009e0 0x7f56b8196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:33.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56befca700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b8101390 0x7f56b81968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:33.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56befca700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81051b0 0x7f56b819ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:33.490 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56befca700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b819b360 con 0x7f56b81051b0 2023-12-14T18:36:33.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56befca700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b819b4d0 con 0x7f56b8101390 2023-12-14T18:36:33.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56befca700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56b819b690 con 0x7f56b81009e0 2023-12-14T18:36:33.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56bd567700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81051b0 0x7f56b819ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:33.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56bd567700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81051b0 0x7f56b819ad20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:44732/0 (socket says 172.21.15.93:44732) 2023-12-14T18:36:33.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56bcd66700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56b81009e0 0x7f56b8196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:33.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56bd567700 1 -- 172.21.15.93:0/986084814 learned_addr learned my addr 172.21.15.93:0/986084814 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:33.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.485+0000 7f56b7fff700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b8101390 0x7f56b81968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:33.491 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56bd567700 1 -- 172.21.15.93:0/986084814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56b81009e0 msgr2=0x7f56b8196380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56bd567700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56b81009e0 0x7f56b8196380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56bd567700 1 -- 172.21.15.93:0/986084814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b8101390 msgr2=0x7f56b81968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:33.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56bd567700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b8101390 0x7f56b81968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:33.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56bd567700 1 -- 172.21.15.93:0/986084814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56a40095f0 con 0x7f56b81051b0 2023-12-14T18:36:33.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56bd567700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81051b0 0x7f56b819ad20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007ee0 tx=0x7f56b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:33.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56b5ffb700 1 -- 172.21.15.93:0/986084814 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b000dd20 con 0x7f56b81051b0 2023-12-14T18:36:33.492 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56b8110320 con 0x7f56b81051b0 2023-12-14T18:36:33.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56b5ffb700 1 -- 172.21.15.93:0/986084814 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f56b000f040 con 0x7f56b81051b0 2023-12-14T18:36:33.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56b5ffb700 1 -- 172.21.15.93:0/986084814 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b00048a0 con 0x7f56b81051b0 2023-12-14T18:36:33.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.486+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56b8110190 con 0x7f56b81051b0 2023-12-14T18:36:33.493 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.487+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56b81104b0 con 0x7f56b81051b0 2023-12-14T18:36:33.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.488+0000 7f56b5ffb700 1 -- 172.21.15.93:0/986084814 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f56b0003a00 con 0x7f56b81051b0 2023-12-14T18:36:33.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.488+0000 7f56b5ffb700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56a80a12c0 0x7f56a80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:33.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.489+0000 7f56b5ffb700 1 -- 172.21.15.93:0/986084814 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 9366+0+0 (secure 0 0 0) 0x7f56b00cd1b0 con 0x7f56b81051b0 2023-12-14T18:36:33.494 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.489+0000 7f56bcd66700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56a80a12c0 0x7f56a80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:33.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.489+0000 7f56bcd66700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56a80a12c0 0x7f56a80a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009be0 tx=0x7f56ac009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:33.495 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.492+0000 7f56b5ffb700 1 -- 172.21.15.93:0/986084814 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f56b0096210 con 0x7f56b81051b0 2023-12-14T18:36:33.737 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.737+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) v1 -- 0x7f56b81a3450 con 0x7f56b81051b0 2023-12-14T18:36:33.745 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.744+0000 7f56b5ffb700 1 -- 172.21.15.93:0/986084814 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v25) v1 ==== 81+0+733775 (secure 0 0 0) 0x7f56b0023090 con 0x7f56b81051b0 2023-12-14T18:36:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:36:33.756 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:36:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: "epoch": 25, 2023-12-14T18:36:33.756 INFO:teuthology.orchestra.run.smithi093.stdout: "active_gid": 25100, 2023-12-14T18:36:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "active_name": "x", 2023-12-14T18:36:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "active_addrs": { 2023-12-14T18:36:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-12-14T18:36:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-12-14T18:36:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:6800", 2023-12-14T18:36:33.757 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3274300600 2023-12-14T18:36:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v1", 2023-12-14T18:36:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:6801", 2023-12-14T18:36:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3274300600 2023-12-14T18:36:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:33.758 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "active_addr": "172.21.15.93:6801/3274300600", 2023-12-14T18:36:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "active_change": "2023-12-14T18:26:27.742571+0000", 2023-12-14T18:36:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "active_mgr_features": 4540138322906710015, 2023-12-14T18:36:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "available": true, 2023-12-14T18:36:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "standbys": [ 2023-12-14T18:36:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "gid": 15471, 2023-12-14T18:36:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "y", 2023-12-14T18:36:33.759 INFO:teuthology.orchestra.run.smithi093.stdout: "mgr_features": 4540138322906710015, 2023-12-14T18:36:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "available_modules": [ 2023-12-14T18:36:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alerts", 2023-12-14T18:36:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:33.760 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.761 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How frequently to reexamine health status", 2023-12-14T18:36:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.762 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.763 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.764 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.765 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:33.766 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.767 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.768 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.769 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.770 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_destination": { 2023-12-14T18:36:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_destination", 2023-12-14T18:36:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.771 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email address to send alerts to", 2023-12-14T18:36:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_from_name": { 2023-12-14T18:36:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_from_name", 2023-12-14T18:36:33.772 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "Ceph", 2023-12-14T18:36:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email From: name", 2023-12-14T18:36:33.773 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_host": { 2023-12-14T18:36:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_host", 2023-12-14T18:36:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.774 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP server", 2023-12-14T18:36:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.775 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_password": { 2023-12-14T18:36:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_password", 2023-12-14T18:36:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.776 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Password to authenticate with", 2023-12-14T18:36:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_port": { 2023-12-14T18:36:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_port", 2023-12-14T18:36:33.777 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "465", 2023-12-14T18:36:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP port", 2023-12-14T18:36:33.778 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_sender": { 2023-12-14T18:36:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_sender", 2023-12-14T18:36:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.779 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP envelope sender", 2023-12-14T18:36:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.780 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_ssl": { 2023-12-14T18:36:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_ssl", 2023-12-14T18:36:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.781 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-14T18:36:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_user": { 2023-12-14T18:36:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_user", 2023-12-14T18:36:33.782 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "User to authenticate as", 2023-12-14T18:36:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.783 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "balancer", 2023-12-14T18:36:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:33.784 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "active": { 2023-12-14T18:36:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "active", 2023-12-14T18:36:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.785 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically balance PGs across cluster", 2023-12-14T18:36:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_time": { 2023-12-14T18:36:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_time", 2023-12-14T18:36:33.786 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0000", 2023-12-14T18:36:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "beginning time of day to automatically balance", 2023-12-14T18:36:33.787 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:36:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_weekday": { 2023-12-14T18:36:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_weekday", 2023-12-14T18:36:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.788 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-12-14T18:36:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-12-14T18:36:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-14T18:36:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:36:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.789 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_max_iterations": { 2023-12-14T18:36:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_max_iterations", 2023-12-14T18:36:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "25", 2023-12-14T18:36:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "250", 2023-12-14T18:36:33.790 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_metrics": { 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_metrics", 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.791 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "pgs,objects,bytes", 2023-12-14T18:36:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-14T18:36:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-14T18:36:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.792 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_step": { 2023-12-14T18:36:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_step", 2023-12-14T18:36:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.5", 2023-12-14T18:36:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0.001", 2023-12-14T18:36:33.793 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "0.999", 2023-12-14T18:36:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "aggressiveness of optimization", 2023-12-14T18:36:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-14T18:36:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "end_time": { 2023-12-14T18:36:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_time", 2023-12-14T18:36:33.794 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2359", 2023-12-14T18:36:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ending time of day to automatically balance", 2023-12-14T18:36:33.795 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:36:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "end_weekday": { 2023-12-14T18:36:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_weekday", 2023-12-14T18:36:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.796 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-12-14T18:36:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-12-14T18:36:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-14T18:36:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:36:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.797 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.798 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.799 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.800 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.801 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.802 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.803 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.804 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.805 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "min_score": { 2023-12-14T18:36:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_score", 2023-12-14T18:36:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.806 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-14T18:36:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-12-14T18:36:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-12-14T18:36:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.807 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "upmap", 2023-12-14T18:36:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "crush-compat", 2023-12-14T18:36:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "none", 2023-12-14T18:36:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap" 2023-12-14T18:36:33.808 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Balancer mode", 2023-12-14T18:36:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_ids": { 2023-12-14T18:36:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_ids", 2023-12-14T18:36:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.809 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-14T18:36:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.810 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.811 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-14T18:36:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_deviation": { 2023-12-14T18:36:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_deviation", 2023-12-14T18:36:33.812 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-14T18:36:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-14T18:36:33.813 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_optimizations": { 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_optimizations", 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.814 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-14T18:36:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.815 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cephadm", 2023-12-14T18:36:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_down_multiplier": { 2023-12-14T18:36:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_down_multiplier", 2023-12-14T18:36:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.816 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-12-14T18:36:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-14T18:36:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.817 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_refresh_rate": { 2023-12-14T18:36:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_refresh_rate", 2023-12-14T18:36:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "20", 2023-12-14T18:36:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.818 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_starting_port": { 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_starting_port", 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.819 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4721", 2023-12-14T18:36:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-14T18:36:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.820 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_password": { 2023-12-14T18:36:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_password", 2023-12-14T18:36:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.821 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web password", 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_user": { 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_user", 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.822 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web user", 2023-12-14T18:36:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.823 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_ptrace": { 2023-12-14T18:36:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_ptrace", 2023-12-14T18:36:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.824 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_interval": { 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_interval", 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.825 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-14T18:36:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.826 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_memory_target_ratio": { 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_memory_target_ratio", 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.7", 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.827 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-14T18:36:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "cgroups_split": { 2023-12-14T18:36:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cgroups_split", 2023-12-14T18:36:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.828 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-14T18:36:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.829 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "config_checks_enabled": { 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_checks_enabled", 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.830 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-14T18:36:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "config_dashboard": { 2023-12-14T18:36:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_dashboard", 2023-12-14T18:36:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.831 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.832 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_alertmanager": { 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_alertmanager", 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.833 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_base": { 2023-12-14T18:36:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_base", 2023-12-14T18:36:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.834 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Container image name, without the tag", 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.835 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_elasticsearch": { 2023-12-14T18:36:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_elasticsearch", 2023-12-14T18:36:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-14T18:36:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.836 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "elasticsearch container image", 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_grafana": { 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_grafana", 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.837 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-14T18:36:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.838 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_haproxy": { 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_haproxy", 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.839 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "HAproxy container image", 2023-12-14T18:36:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_agent": { 2023-12-14T18:36:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_agent", 2023-12-14T18:36:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.840 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-14T18:36:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger agent container image", 2023-12-14T18:36:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.841 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_collector": { 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_collector", 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.842 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger collector container image", 2023-12-14T18:36:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_query": { 2023-12-14T18:36:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_query", 2023-12-14T18:36:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.843 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger query container image", 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.844 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_keepalived": { 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_keepalived", 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Keepalived container image", 2023-12-14T18:36:33.845 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_loki": { 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_loki", 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-14T18:36:33.846 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Loki container image", 2023-12-14T18:36:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_node_exporter": { 2023-12-14T18:36:33.847 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_node_exporter", 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.848 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_prometheus": { 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_prometheus", 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.849 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_promtail": { 2023-12-14T18:36:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_promtail", 2023-12-14T18:36:33.850 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Promtail container image", 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.851 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_snmp_gateway": { 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_snmp_gateway", 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.852 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SNMP Gateway container image", 2023-12-14T18:36:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "container_init": { 2023-12-14T18:36:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_init", 2023-12-14T18:36:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.853 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Run podman/docker with `--init`", 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.854 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "daemon_cache_timeout": { 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "daemon_cache_timeout", 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.855 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "default_cephadm_command_timeout": { 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_cephadm_command_timeout", 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.856 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "900", 2023-12-14T18:36:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-14T18:36:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.857 INFO:teuthology.orchestra.run.smithi093.stdout: "default_registry": { 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_registry", 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io", 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-14T18:36:33.858 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "device_cache_timeout": { 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_cache_timeout", 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1800", 2023-12-14T18:36:33.859 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache device inventory", 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "device_enhanced_scan": { 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_enhanced_scan", 2023-12-14T18:36:33.860 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.861 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "facts_cache_timeout": { 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "facts_cache_timeout", 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.862 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache host facts data", 2023-12-14T18:36:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "host_check_interval": { 2023-12-14T18:36:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "host_check_interval", 2023-12-14T18:36:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.863 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to perform a host check", 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.864 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.865 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.866 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "log_refresh_metadata": { 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_refresh_metadata", 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.867 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.868 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-14T18:36:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.869 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.870 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.871 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.872 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.873 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-14T18:36:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.874 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "*", 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.875 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "max_count_per_host": { 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_count_per_host", 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.876 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of daemons per service per host", 2023-12-14T18:36:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.877 INFO:teuthology.orchestra.run.smithi093.stdout: "max_osd_draining_count": { 2023-12-14T18:36:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_osd_draining_count", 2023-12-14T18:36:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.878 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "migration_current": { 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "migration_current", 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.879 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "internal - do not modify", 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-12-14T18:36:33.880 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "root", 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm-package", 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: "root" 2023-12-14T18:36:33.881 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "mode for remote execution of cephadm", 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_alerts_path": { 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_alerts_path", 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.882 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.883 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_password": { 2023-12-14T18:36:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_password", 2023-12-14T18:36:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.884 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web password", 2023-12-14T18:36:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_user": { 2023-12-14T18:36:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_user", 2023-12-14T18:36:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.885 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web user", 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.886 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_insecure": { 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_insecure", 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-14T18:36:33.887 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_password": { 2023-12-14T18:36:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_password", 2023-12-14T18:36:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.888 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-14T18:36:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.889 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_url": { 2023-12-14T18:36:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_url", 2023-12-14T18:36:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.890 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-14T18:36:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_username": { 2023-12-14T18:36:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_username", 2023-12-14T18:36:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.891 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.892 INFO:teuthology.orchestra.run.smithi093.stdout: "secure_monitoring_stack": { 2023-12-14T18:36:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "secure_monitoring_stack", 2023-12-14T18:36:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.893 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "service_discovery_port": { 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "service_discovery_port", 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.894 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8765", 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "cephadm service discovery port", 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "ssh_config_file": { 2023-12-14T18:36:33.895 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssh_config_file", 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.896 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "use_agent": { 2023-12-14T18:36:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_agent", 2023-12-14T18:36:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.897 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "use_repo_digest": { 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_repo_digest", 2023-12-14T18:36:33.898 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.899 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_failed_host_check": { 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_failed_host_check", 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.900 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if the host check fails", 2023-12-14T18:36:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_daemons": { 2023-12-14T18:36:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_daemons", 2023-12-14T18:36:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.901 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.902 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_hosts": { 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_hosts", 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.903 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crash", 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:33.904 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.905 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.906 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.907 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:33.908 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.909 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.910 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.911 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "retain_interval": { 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retain_interval", 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "31536000", 2023-12-14T18:36:33.912 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-14T18:36:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_recent_interval": { 2023-12-14T18:36:33.913 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_recent_interval", 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1209600", 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.914 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dashboard", 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:33.915 INFO:teuthology.orchestra.run.smithi093.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.916 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_HOST": { 2023-12-14T18:36:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-14T18:36:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.917 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-14T18:36:33.918 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.919 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_ENABLED": { 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_ENABLED", 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.920 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-14T18:36:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-14T18:36:33.921 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.922 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "ENABLE_BROWSABLE_API": { 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.923 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-14T18:36:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-14T18:36:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.924 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.925 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.926 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.927 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.928 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.929 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-14T18:36:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-14T18:36:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.930 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-14T18:36:33.931 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.932 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.933 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-14T18:36:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-14T18:36:33.934 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.935 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_PASSWORD": { 2023-12-14T18:36:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-14T18:36:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.936 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.937 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.938 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_URL": { 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_URL", 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.939 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_USERNAME": { 2023-12-14T18:36:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-14T18:36:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.940 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-14T18:36:33.941 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.942 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.943 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-14T18:36:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-14T18:36:33.944 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.945 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.946 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_HOST": { 2023-12-14T18:36:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-14T18:36:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.947 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.948 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-14T18:36:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-14T18:36:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.949 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.950 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.951 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.952 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.953 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-14T18:36:33.954 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.955 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.956 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-14T18:36:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-14T18:36:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.957 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.958 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.959 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_ENABLED": { 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.960 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.961 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.962 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:33.963 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-14T18:36:33.964 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8", 2023-12-14T18:36:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.965 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "45", 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.966 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ACCESS_KEY": { 2023-12-14T18:36:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-14T18:36:33.967 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.968 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.969 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SECRET_KEY": { 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.970 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.971 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SSL_VERIFY": { 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.972 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:33.973 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-14T18:36:33.974 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.975 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.976 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "cross_origin_url": { 2023-12-14T18:36:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cross_origin_url", 2023-12-14T18:36:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.977 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.978 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "crt_file": { 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crt_file", 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.979 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "debug": { 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "debug", 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.980 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable/disable debug options", 2023-12-14T18:36:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.981 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "jwt_token_ttl": { 2023-12-14T18:36:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "jwt_token_ttl", 2023-12-14T18:36:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "28800", 2023-12-14T18:36:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.982 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.983 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.984 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.985 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.986 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.987 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.988 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:33.989 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:33.990 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.991 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "motd": { 2023-12-14T18:36:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "motd", 2023-12-14T18:36:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:33.992 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "The message of the day", 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect_resolve_ip_addr": { 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "redirect_resolve_ip_addr", 2023-12-14T18:36:33.993 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.994 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.995 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.996 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8080", 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.997 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-12-14T18:36:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-12-14T18:36:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:33.998 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl_server_port": { 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl_server_port", 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:33.999 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8443", 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.000 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-12-14T18:36:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-12-14T18:36:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "redirect", 2023-12-14T18:36:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.001 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect" 2023-12-14T18:36:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-12-14T18:36:34.002 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.003 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "url_prefix": { 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url_prefix", 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.004 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.005 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-12-14T18:36:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_monitoring": { 2023-12-14T18:36:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_monitoring", 2023-12-14T18:36:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.006 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "monitor device health metrics", 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.007 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.008 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.009 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.010 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.011 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.012 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.013 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.014 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "mark_out_threshold": { 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mark_out_threshold", 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2419200", 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.015 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_name": { 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_name", 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.016 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "device_health_metrics", 2023-12-14T18:36:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-14T18:36:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.017 INFO:teuthology.orchestra.run.smithi093.stdout: "retention_period": { 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retention_period", 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15552000", 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain device health metrics", 2023-12-14T18:36:34.018 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_frequency": { 2023-12-14T18:36:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_frequency", 2023-12-14T18:36:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.019 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "self_heal": { 2023-12-14T18:36:34.020 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "self_heal", 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.021 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.022 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and check device health", 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_threshold": { 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_threshold", 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.023 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7257600", 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.024 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "diskprediction_local", 2023-12-14T18:36:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.025 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.026 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.027 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.028 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.029 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.030 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.031 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.032 INFO:teuthology.orchestra.run.smithi093.stdout: "predict_interval": { 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predict_interval", 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.033 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "predictor_model": { 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predictor_model", 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "prophetstor", 2023-12-14T18:36:34.034 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:34.035 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.036 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "influx", 2023-12-14T18:36:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": false, 2023-12-14T18:36:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "influxdb python module not found", 2023-12-14T18:36:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.037 INFO:teuthology.orchestra.run.smithi093.stdout: "batch_size": { 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "batch_size", 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5000", 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-14T18:36:34.038 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "database": { 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "database", 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "ceph", 2023-12-14T18:36:34.039 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": { 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "hostname", 2023-12-14T18:36:34.040 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server hostname", 2023-12-14T18:36:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.041 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "30", 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "5", 2023-12-14T18:36:34.042 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.043 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.044 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.045 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.046 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.047 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.048 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.049 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "password": { 2023-12-14T18:36:34.050 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "password", 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "password of InfluxDB server user", 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.051 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "port": { 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "port", 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8086", 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.052 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server port", 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.053 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "false", 2023-12-14T18:36:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:36:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.054 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "threads": { 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threads", 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "32", 2023-12-14T18:36:34.055 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "username": { 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "username", 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.056 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "username of InfluxDB server user", 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.057 INFO:teuthology.orchestra.run.smithi093.stdout: "verify_ssl": { 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "verify_ssl", 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "true", 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:36:34.058 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "insights", 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.059 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.060 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.061 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.062 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.063 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.064 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.065 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.066 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "iostat", 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.067 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.068 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.069 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.070 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.071 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.072 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.073 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.074 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "k8sevents", 2023-12-14T18:36:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.075 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_event_retention_days": { 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ceph_event_retention_days", 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7", 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-14T18:36:34.076 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "config_check_secs": { 2023-12-14T18:36:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_check_secs", 2023-12-14T18:36:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.077 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "10", 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.078 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.079 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.080 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.081 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.082 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.083 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.084 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.085 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "localpool", 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.086 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "failure_domain": { 2023-12-14T18:36:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "failure_domain", 2023-12-14T18:36:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "host", 2023-12-14T18:36:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.087 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "failure domain for any created local pool", 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.088 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.089 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.090 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.091 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.092 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.093 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.094 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "min_size": { 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_size", 2023-12-14T18:36:34.095 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default min_size for any created local pool", 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.096 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "num_rep": { 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "num_rep", 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3", 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.097 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default replica count for any created local pool", 2023-12-14T18:36:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_num": { 2023-12-14T18:36:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_num", 2023-12-14T18:36:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.098 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "128", 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default pg_num for any created local pool", 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.099 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "prefix": { 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prefix", 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.100 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name prefix for any created local pool", 2023-12-14T18:36:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "subtree": { 2023-12-14T18:36:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "subtree", 2023-12-14T18:36:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.101 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "rack", 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.102 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mds_autoscaler", 2023-12-14T18:36:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.103 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.104 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.105 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.106 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.107 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.108 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.109 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.110 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirroring", 2023-12-14T18:36:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.111 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.112 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.113 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.114 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.115 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.116 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.117 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.118 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "nfs", 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.119 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.120 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.121 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.122 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.123 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.124 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.125 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.126 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "fail_fs": { 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "fail_fs", 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.127 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-14T18:36:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.128 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.129 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.130 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.131 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.132 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.133 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.134 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.135 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator": { 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-12-14T18:36:34.136 INFO:teuthology.orchestra.run.smithi093.stdout: "rook", 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: "test_orchestrator" 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Orchestrator backend", 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.137 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_perf_query", 2023-12-14T18:36:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.138 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.139 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.140 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.141 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.142 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.143 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.144 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.145 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_support", 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.146 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.147 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.148 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.149 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.150 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.151 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.152 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_autoscaler", 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.153 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.154 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.155 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.156 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.157 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.158 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.159 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.160 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "noautoscale": { 2023-12-14T18:36:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "noautoscale", 2023-12-14T18:36:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.161 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "global autoscale flag", 2023-12-14T18:36:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-12-14T18:36:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:34.162 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.163 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "threshold": { 2023-12-14T18:36:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threshold", 2023-12-14T18:36:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-12-14T18:36:34.164 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1.0", 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "scaling threshold", 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.165 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "progress", 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_pg_recovery_event": { 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_pg_recovery_event", 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.166 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-12-14T18:36:34.167 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.168 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.169 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.170 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.171 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.172 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.173 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.174 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.175 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "max_completed_events": { 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_completed_events", 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "50", 2023-12-14T18:36:34.176 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "number of past completed events to remember", 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:34.177 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long the module is going to sleep", 2023-12-14T18:36:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.178 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus", 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.179 INFO:teuthology.orchestra.run.smithi093.stdout: "cache": { 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cache", 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.180 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "exclude_perf_counters": { 2023-12-14T18:36:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "exclude_perf_counters", 2023-12-14T18:36:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.181 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.182 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.183 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.184 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.185 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.186 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.187 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.188 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.189 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools": { 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools", 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.190 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.191 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300", 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.192 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_interval": { 2023-12-14T18:36:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_interval", 2023-12-14T18:36:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15.0", 2023-12-14T18:36:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.193 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-12-14T18:36:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-12-14T18:36:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.194 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-12-14T18:36:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-14T18:36:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.195 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-12-14T18:36:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-12-14T18:36:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "9283", 2023-12-14T18:36:34.196 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-14T18:36:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.197 INFO:teuthology.orchestra.run.smithi093.stdout: "stale_cache_strategy": { 2023-12-14T18:36:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stale_cache_strategy", 2023-12-14T18:36:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "log", 2023-12-14T18:36:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.198 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.199 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "default", 2023-12-14T18:36:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "default", 2023-12-14T18:36:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "error" 2023-12-14T18:36:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.200 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-12-14T18:36:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-12-14T18:36:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-12-14T18:36:34.201 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-12-14T18:36:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-12-14T18:36:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.202 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-12-14T18:36:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.203 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.204 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.205 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.206 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.207 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.208 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.209 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.210 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_snap_create": { 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_snap_create", 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.211 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "mirror_snapshot_schedule": { 2023-12-14T18:36:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirror_snapshot_schedule", 2023-12-14T18:36:34.212 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.213 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "trash_purge_schedule": { 2023-12-14T18:36:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "trash_purge_schedule", 2023-12-14T18:36:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.214 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "restful", 2023-12-14T18:36:34.215 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_auth": { 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_auth", 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.216 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-12-14T18:36:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-12-14T18:36:34.217 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.218 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.219 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.220 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.221 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.222 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.223 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.224 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.225 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.226 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.227 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-12-14T18:36:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-12-14T18:36:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.228 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.229 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rgw", 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.230 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.231 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.232 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.233 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.234 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.235 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.236 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rook", 2023-12-14T18:36:34.237 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "drive_group_interval": { 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "drive_group_interval", 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300.0", 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.238 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.239 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.240 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.241 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.242 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.243 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.244 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.245 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.246 INFO:teuthology.orchestra.run.smithi093.stdout: "storage_class": { 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "storage_class", 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "local", 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-14T18:36:34.247 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "selftest", 2023-12-14T18:36:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.248 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.249 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.250 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.251 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.252 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.253 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.254 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.255 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "roption1": { 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption1", 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.256 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.257 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "roption2": { 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption2", 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "xyz", 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.258 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption1": { 2023-12-14T18:36:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption1", 2023-12-14T18:36:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.259 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.260 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption2": { 2023-12-14T18:36:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption2", 2023-12-14T18:36:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.261 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption3": { 2023-12-14T18:36:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption3", 2023-12-14T18:36:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.262 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.263 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption4": { 2023-12-14T18:36:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption4", 2023-12-14T18:36:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.264 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption5": { 2023-12-14T18:36:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption5", 2023-12-14T18:36:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.265 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.266 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption6": { 2023-12-14T18:36:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption6", 2023-12-14T18:36:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.267 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption7": { 2023-12-14T18:36:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption7", 2023-12-14T18:36:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.268 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "42", 2023-12-14T18:36:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.269 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "testkey": { 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testkey", 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.270 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "testlkey": { 2023-12-14T18:36:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testlkey", 2023-12-14T18:36:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.271 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.272 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "testnewline": { 2023-12-14T18:36:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testnewline", 2023-12-14T18:36:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.273 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.274 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_m_granularity": { 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_m_granularity", 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.275 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow minute scheduled snapshots", 2023-12-14T18:36:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "dump_on_update": { 2023-12-14T18:36:34.276 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dump_on_update", 2023-12-14T18:36:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "dump database to debug log on update", 2023-12-14T18:36:34.277 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.278 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.279 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.280 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.281 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.282 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.283 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.284 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.285 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stats", 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.286 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.287 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.288 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.289 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.290 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.291 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.292 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.293 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "status", 2023-12-14T18:36:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.294 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.295 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.296 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.297 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.298 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.299 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.300 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.301 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.302 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telegraf", 2023-12-14T18:36:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "address": { 2023-12-14T18:36:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "address", 2023-12-14T18:36:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.303 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.304 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15", 2023-12-14T18:36:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.305 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.306 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.307 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.308 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.309 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.310 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.311 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.312 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.313 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telemetry", 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_basic": { 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_basic", 2023-12-14T18:36:34.314 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.315 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_crash": { 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_crash", 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.316 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-14T18:36:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_device": { 2023-12-14T18:36:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_device", 2023-12-14T18:36:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.317 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.318 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_ident": { 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_ident", 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.319 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-14T18:36:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_perf": { 2023-12-14T18:36:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_perf", 2023-12-14T18:36:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.320 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.321 INFO:teuthology.orchestra.run.smithi093.stdout: "contact": { 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "contact", 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.322 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "description": { 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "description", 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.323 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "device_url": { 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_url", 2023-12-14T18:36:34.324 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.325 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-12-14T18:36:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-12-14T18:36:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.326 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.327 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "24", 2023-12-14T18:36:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "8", 2023-12-14T18:36:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.328 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "last_opt_revision": { 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "last_opt_revision", 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1", 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.329 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard": { 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard", 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.330 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.331 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard_description": { 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard_description", 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.332 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.333 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.334 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.335 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.336 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.337 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.338 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.339 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "organization": { 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "organization", 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.340 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.341 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "proxy": { 2023-12-14T18:36:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "proxy", 2023-12-14T18:36:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.342 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "url": { 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url", 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.343 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-14T18:36:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.344 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "test_orchestrator", 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.345 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.346 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.347 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.348 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.349 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.350 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.351 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.352 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.353 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.354 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.355 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.356 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.357 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.358 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.359 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.360 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_clones": { 2023-12-14T18:36:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_clones", 2023-12-14T18:36:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4", 2023-12-14T18:36:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.361 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "snapshot_clone_delay": { 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snapshot_clone_delay", 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.362 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.363 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix", 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "discovery_interval": { 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "discovery_interval", 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:34.364 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "100", 2023-12-14T18:36:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.365 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "identifier": { 2023-12-14T18:36:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "identifier", 2023-12-14T18:36:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.366 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.367 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.368 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.369 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.370 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.371 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.372 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.373 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.374 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.375 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_host": { 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_host", 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.376 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_port": { 2023-12-14T18:36:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_port", 2023-12-14T18:36:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.377 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10051", 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.378 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_sender": { 2023-12-14T18:36:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_sender", 2023-12-14T18:36:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-14T18:36:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.379 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.380 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "gid": 25096, 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "z", 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "mgr_features": 4540138322906710015, 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "available_modules": [ 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alerts", 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.381 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How frequently to reexamine health status", 2023-12-14T18:36:34.382 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.383 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.384 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.385 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.386 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.387 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.388 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.389 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_destination": { 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_destination", 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.390 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email address to send alerts to", 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.391 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_from_name": { 2023-12-14T18:36:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_from_name", 2023-12-14T18:36:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "Ceph", 2023-12-14T18:36:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.392 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email From: name", 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_host": { 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_host", 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.393 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP server", 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_password": { 2023-12-14T18:36:34.394 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_password", 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Password to authenticate with", 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.395 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_port": { 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_port", 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "465", 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.396 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP port", 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_sender": { 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_sender", 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.397 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP envelope sender", 2023-12-14T18:36:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.398 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_ssl": { 2023-12-14T18:36:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_ssl", 2023-12-14T18:36:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.399 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_user": { 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_user", 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.400 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "User to authenticate as", 2023-12-14T18:36:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.401 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "balancer", 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "active": { 2023-12-14T18:36:34.402 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "active", 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically balance PGs across cluster", 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.403 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_time": { 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_time", 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0000", 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.404 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "beginning time of day to automatically balance", 2023-12-14T18:36:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:36:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_weekday": { 2023-12-14T18:36:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_weekday", 2023-12-14T18:36:34.405 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-12-14T18:36:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-12-14T18:36:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-14T18:36:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:36:34.406 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_max_iterations": { 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_max_iterations", 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "25", 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:34.407 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "250", 2023-12-14T18:36:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-14T18:36:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_metrics": { 2023-12-14T18:36:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_metrics", 2023-12-14T18:36:34.408 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "pgs,objects,bytes", 2023-12-14T18:36:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-14T18:36:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-14T18:36:34.409 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_step": { 2023-12-14T18:36:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_step", 2023-12-14T18:36:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.413 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.5", 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0.001", 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "0.999", 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "aggressiveness of optimization", 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "end_time": { 2023-12-14T18:36:34.414 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_time", 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2359", 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ending time of day to automatically balance", 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:36:34.415 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "end_weekday": { 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_weekday", 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-12-14T18:36:34.416 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-12-14T18:36:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-14T18:36:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:36:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.417 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.418 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.419 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.420 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.421 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.422 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.423 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.424 INFO:teuthology.orchestra.run.smithi093.stdout: "min_score": { 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_score", 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-14T18:36:34.425 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-12-14T18:36:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-12-14T18:36:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.426 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "upmap", 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "crush-compat", 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "none", 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap" 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Balancer mode", 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.427 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_ids": { 2023-12-14T18:36:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_ids", 2023-12-14T18:36:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.428 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:34.429 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-14T18:36:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.430 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_deviation": { 2023-12-14T18:36:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_deviation", 2023-12-14T18:36:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:34.431 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-14T18:36:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-14T18:36:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_optimizations": { 2023-12-14T18:36:34.432 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_optimizations", 2023-12-14T18:36:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-14T18:36:34.433 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cephadm", 2023-12-14T18:36:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.434 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_down_multiplier": { 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_down_multiplier", 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.435 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-14T18:36:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_refresh_rate": { 2023-12-14T18:36:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_refresh_rate", 2023-12-14T18:36:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.436 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "20", 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.437 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_starting_port": { 2023-12-14T18:36:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_starting_port", 2023-12-14T18:36:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4721", 2023-12-14T18:36:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.438 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_password": { 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_password", 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.439 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web password", 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.440 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_user": { 2023-12-14T18:36:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_user", 2023-12-14T18:36:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.441 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web user", 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_ptrace": { 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_ptrace", 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.442 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_interval": { 2023-12-14T18:36:34.443 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_interval", 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.444 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_memory_target_ratio": { 2023-12-14T18:36:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_memory_target_ratio", 2023-12-14T18:36:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.7", 2023-12-14T18:36:34.445 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "cgroups_split": { 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cgroups_split", 2023-12-14T18:36:34.446 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.447 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "config_checks_enabled": { 2023-12-14T18:36:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_checks_enabled", 2023-12-14T18:36:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.448 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-14T18:36:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "config_dashboard": { 2023-12-14T18:36:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_dashboard", 2023-12-14T18:36:34.449 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.450 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_alertmanager": { 2023-12-14T18:36:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_alertmanager", 2023-12-14T18:36:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-14T18:36:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.451 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_base": { 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_base", 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.452 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-14T18:36:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Container image name, without the tag", 2023-12-14T18:36:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.453 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_elasticsearch": { 2023-12-14T18:36:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_elasticsearch", 2023-12-14T18:36:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-14T18:36:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.454 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "elasticsearch container image", 2023-12-14T18:36:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_grafana": { 2023-12-14T18:36:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_grafana", 2023-12-14T18:36:34.455 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.456 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_haproxy": { 2023-12-14T18:36:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_haproxy", 2023-12-14T18:36:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-14T18:36:34.457 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "HAproxy container image", 2023-12-14T18:36:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_agent": { 2023-12-14T18:36:34.458 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_agent", 2023-12-14T18:36:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-14T18:36:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger agent container image", 2023-12-14T18:36:34.459 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_collector": { 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_collector", 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-14T18:36:34.460 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger collector container image", 2023-12-14T18:36:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_query": { 2023-12-14T18:36:34.461 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_query", 2023-12-14T18:36:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-14T18:36:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger query container image", 2023-12-14T18:36:34.462 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_keepalived": { 2023-12-14T18:36:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_keepalived", 2023-12-14T18:36:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.463 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-14T18:36:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Keepalived container image", 2023-12-14T18:36:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.464 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_loki": { 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_loki", 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Loki container image", 2023-12-14T18:36:34.465 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_node_exporter": { 2023-12-14T18:36:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_node_exporter", 2023-12-14T18:36:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.466 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-14T18:36:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.467 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_prometheus": { 2023-12-14T18:36:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_prometheus", 2023-12-14T18:36:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-14T18:36:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.468 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_promtail": { 2023-12-14T18:36:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_promtail", 2023-12-14T18:36:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.469 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-14T18:36:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Promtail container image", 2023-12-14T18:36:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.470 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_snmp_gateway": { 2023-12-14T18:36:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_snmp_gateway", 2023-12-14T18:36:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-14T18:36:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.471 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SNMP Gateway container image", 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "container_init": { 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_init", 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.472 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Run podman/docker with `--init`", 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.473 INFO:teuthology.orchestra.run.smithi093.stdout: "daemon_cache_timeout": { 2023-12-14T18:36:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "daemon_cache_timeout", 2023-12-14T18:36:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.474 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "default_cephadm_command_timeout": { 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_cephadm_command_timeout", 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.475 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "900", 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "default_registry": { 2023-12-14T18:36:34.476 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_registry", 2023-12-14T18:36:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io", 2023-12-14T18:36:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-14T18:36:34.477 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "device_cache_timeout": { 2023-12-14T18:36:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_cache_timeout", 2023-12-14T18:36:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.478 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1800", 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache device inventory", 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "device_enhanced_scan": { 2023-12-14T18:36:34.479 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_enhanced_scan", 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.480 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "facts_cache_timeout": { 2023-12-14T18:36:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "facts_cache_timeout", 2023-12-14T18:36:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:34.481 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache host facts data", 2023-12-14T18:36:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "host_check_interval": { 2023-12-14T18:36:34.482 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "host_check_interval", 2023-12-14T18:36:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to perform a host check", 2023-12-14T18:36:34.483 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.484 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.485 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "log_refresh_metadata": { 2023-12-14T18:36:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_refresh_metadata", 2023-12-14T18:36:34.486 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-14T18:36:34.487 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.488 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-14T18:36:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.489 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.490 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.491 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.492 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.493 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-14T18:36:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.494 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-14T18:36:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-14T18:36:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "*", 2023-12-14T18:36:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.495 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "max_count_per_host": { 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_count_per_host", 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.496 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of daemons per service per host", 2023-12-14T18:36:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.497 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "max_osd_draining_count": { 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_osd_draining_count", 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.498 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-14T18:36:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "migration_current": { 2023-12-14T18:36:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "migration_current", 2023-12-14T18:36:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.499 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "internal - do not modify", 2023-12-14T18:36:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.500 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "root", 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.501 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm-package", 2023-12-14T18:36:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "root" 2023-12-14T18:36:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "mode for remote execution of cephadm", 2023-12-14T18:36:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.502 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_alerts_path": { 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_alerts_path", 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-14T18:36:34.503 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_password": { 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_password", 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:34.504 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web password", 2023-12-14T18:36:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_user": { 2023-12-14T18:36:34.505 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_user", 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web user", 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.506 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_insecure": { 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_insecure", 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.507 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-14T18:36:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_password": { 2023-12-14T18:36:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_password", 2023-12-14T18:36:34.508 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.509 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_url": { 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_url", 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.510 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-14T18:36:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_username": { 2023-12-14T18:36:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_username", 2023-12-14T18:36:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.511 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.512 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "secure_monitoring_stack": { 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "secure_monitoring_stack", 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.513 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "service_discovery_port": { 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "service_discovery_port", 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.514 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8765", 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "cephadm service discovery port", 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "ssh_config_file": { 2023-12-14T18:36:34.515 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssh_config_file", 2023-12-14T18:36:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-14T18:36:34.516 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "use_agent": { 2023-12-14T18:36:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_agent", 2023-12-14T18:36:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.517 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-14T18:36:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.518 INFO:teuthology.orchestra.run.smithi093.stdout: "use_repo_digest": { 2023-12-14T18:36:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_repo_digest", 2023-12-14T18:36:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.519 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-14T18:36:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_failed_host_check": { 2023-12-14T18:36:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_failed_host_check", 2023-12-14T18:36:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.520 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if the host check fails", 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.521 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_daemons": { 2023-12-14T18:36:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_daemons", 2023-12-14T18:36:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.522 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_hosts": { 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_hosts", 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.523 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.524 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crash", 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.525 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.526 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.527 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.528 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.529 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.530 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.531 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "retain_interval": { 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retain_interval", 2023-12-14T18:36:34.532 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "31536000", 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.533 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_recent_interval": { 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_recent_interval", 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1209600", 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.534 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-14T18:36:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.535 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dashboard", 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.536 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_HOST": { 2023-12-14T18:36:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-14T18:36:34.537 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.538 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.539 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_ENABLED": { 2023-12-14T18:36:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_ENABLED", 2023-12-14T18:36:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.540 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.541 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.542 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "ENABLE_BROWSABLE_API": { 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.543 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-14T18:36:34.544 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-14T18:36:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.545 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.546 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-14T18:36:34.547 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.548 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.549 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.550 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.551 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.552 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.553 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-14T18:36:34.554 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.555 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_PASSWORD": { 2023-12-14T18:36:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-14T18:36:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.556 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.557 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.558 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_URL": { 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_URL", 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.559 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_USERNAME": { 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.560 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.561 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.562 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.563 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-14T18:36:34.564 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.565 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.566 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_HOST": { 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.567 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.568 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.569 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.570 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-14T18:36:34.571 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.572 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.573 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.574 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.575 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.576 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-14T18:36:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-14T18:36:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.577 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.578 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.579 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_ENABLED": { 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.580 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.581 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.582 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.583 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-14T18:36:34.584 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8", 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.585 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-14T18:36:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-14T18:36:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "45", 2023-12-14T18:36:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.586 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ACCESS_KEY": { 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.587 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.588 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-14T18:36:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-14T18:36:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.589 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SECRET_KEY": { 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.590 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.591 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SSL_VERIFY": { 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.592 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:34.593 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-14T18:36:34.594 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.595 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.596 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "cross_origin_url": { 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cross_origin_url", 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.597 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.598 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "crt_file": { 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crt_file", 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.599 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "debug": { 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "debug", 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.600 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable/disable debug options", 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.601 INFO:teuthology.orchestra.run.smithi093.stdout: "jwt_token_ttl": { 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "jwt_token_ttl", 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "28800", 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.602 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.603 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.604 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.605 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.606 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.607 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.608 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.609 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.610 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.611 INFO:teuthology.orchestra.run.smithi093.stdout: "motd": { 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "motd", 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "The message of the day", 2023-12-14T18:36:34.612 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect_resolve_ip_addr": { 2023-12-14T18:36:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "redirect_resolve_ip_addr", 2023-12-14T18:36:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.613 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.614 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.615 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-12-14T18:36:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-12-14T18:36:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.616 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8080", 2023-12-14T18:36:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.617 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-12-14T18:36:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-12-14T18:36:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.618 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl_server_port": { 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl_server_port", 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.619 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8443", 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-12-14T18:36:34.620 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "redirect", 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect" 2023-12-14T18:36:34.621 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.622 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-12-14T18:36:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-12-14T18:36:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-12-14T18:36:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.623 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "url_prefix": { 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url_prefix", 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.624 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-12-14T18:36:34.625 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_monitoring": { 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_monitoring", 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.626 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "monitor device health metrics", 2023-12-14T18:36:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.627 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.628 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.629 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.630 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.631 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.632 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.633 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.634 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "mark_out_threshold": { 2023-12-14T18:36:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mark_out_threshold", 2023-12-14T18:36:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2419200", 2023-12-14T18:36:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.635 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_name": { 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_name", 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.636 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "device_health_metrics", 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.637 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "retention_period": { 2023-12-14T18:36:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retention_period", 2023-12-14T18:36:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15552000", 2023-12-14T18:36:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.638 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain device health metrics", 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_frequency": { 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_frequency", 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.639 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.640 INFO:teuthology.orchestra.run.smithi093.stdout: "self_heal": { 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "self_heal", 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-14T18:36:34.641 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:34.642 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and check device health", 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_threshold": { 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_threshold", 2023-12-14T18:36:34.643 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7257600", 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.644 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "diskprediction_local", 2023-12-14T18:36:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.645 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.646 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.647 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.648 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.649 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.650 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.651 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.652 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "predict_interval": { 2023-12-14T18:36:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predict_interval", 2023-12-14T18:36:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-12-14T18:36:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.653 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "predictor_model": { 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predictor_model", 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.654 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "prophetstor", 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.655 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.656 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "influx", 2023-12-14T18:36:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": false, 2023-12-14T18:36:34.657 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "influxdb python module not found", 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "batch_size": { 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "batch_size", 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5000", 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.658 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "database": { 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "database", 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.659 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "ceph", 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.660 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": { 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "hostname", 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server hostname", 2023-12-14T18:36:34.661 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "30", 2023-12-14T18:36:34.662 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "5", 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.663 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.664 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.665 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.666 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.667 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.668 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.669 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.670 INFO:teuthology.orchestra.run.smithi093.stdout: "password": { 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "password", 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "password of InfluxDB server user", 2023-12-14T18:36:34.671 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "port": { 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "port", 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8086", 2023-12-14T18:36:34.672 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server port", 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-12-14T18:36:34.673 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "false", 2023-12-14T18:36:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:36:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.674 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "threads": { 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threads", 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:34.675 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "32", 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "username": { 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "username", 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.676 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "username of InfluxDB server user", 2023-12-14T18:36:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.677 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "verify_ssl": { 2023-12-14T18:36:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "verify_ssl", 2023-12-14T18:36:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "true", 2023-12-14T18:36:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.678 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "insights", 2023-12-14T18:36:34.679 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.680 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.681 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.682 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.683 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.684 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.685 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.686 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "iostat", 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.687 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.688 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.689 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.690 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.691 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.692 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.693 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.694 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "k8sevents", 2023-12-14T18:36:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.695 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_event_retention_days": { 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ceph_event_retention_days", 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7", 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.696 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "config_check_secs": { 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_check_secs", 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.697 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "10", 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.698 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.699 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.700 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.701 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.702 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.703 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.704 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.705 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "localpool", 2023-12-14T18:36:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.706 INFO:teuthology.orchestra.run.smithi093.stdout: "failure_domain": { 2023-12-14T18:36:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "failure_domain", 2023-12-14T18:36:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "host", 2023-12-14T18:36:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.707 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "failure domain for any created local pool", 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.708 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.709 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.710 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.711 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.712 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.713 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.714 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "min_size": { 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_size", 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.715 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default min_size for any created local pool", 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.716 INFO:teuthology.orchestra.run.smithi093.stdout: "num_rep": { 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "num_rep", 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3", 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default replica count for any created local pool", 2023-12-14T18:36:34.717 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_num": { 2023-12-14T18:36:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_num", 2023-12-14T18:36:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.718 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "128", 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default pg_num for any created local pool", 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "prefix": { 2023-12-14T18:36:34.719 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prefix", 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name prefix for any created local pool", 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.720 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "subtree": { 2023-12-14T18:36:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "subtree", 2023-12-14T18:36:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "rack", 2023-12-14T18:36:34.721 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.722 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mds_autoscaler", 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.723 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.724 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.725 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.726 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.727 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.728 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.729 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.730 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirroring", 2023-12-14T18:36:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.731 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.732 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.733 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.734 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.735 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.736 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.737 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.738 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "nfs", 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.739 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.740 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.741 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.742 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.743 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.744 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.745 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-12-14T18:36:34.746 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "fail_fs": { 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "fail_fs", 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.747 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.748 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.749 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.750 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.751 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.752 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.753 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.754 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator": { 2023-12-14T18:36:34.755 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-12-14T18:36:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-12-14T18:36:34.756 INFO:teuthology.orchestra.run.smithi093.stdout: "rook", 2023-12-14T18:36:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "test_orchestrator" 2023-12-14T18:36:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Orchestrator backend", 2023-12-14T18:36:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.757 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_perf_query", 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.758 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.759 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.760 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.761 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.762 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.763 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.764 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.765 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_support", 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.766 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.767 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.768 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.769 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.770 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.771 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.772 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.773 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_autoscaler", 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.774 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.775 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.776 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.777 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.778 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.779 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.780 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "noautoscale": { 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "noautoscale", 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.781 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "global autoscale flag", 2023-12-14T18:36:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-12-14T18:36:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.782 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.783 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "threshold": { 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threshold", 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-12-14T18:36:34.784 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1.0", 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "scaling threshold", 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.785 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "progress", 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_pg_recovery_event": { 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_pg_recovery_event", 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.786 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-14T18:36:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.787 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.788 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.789 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.790 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.791 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.792 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.793 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.794 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.795 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.796 INFO:teuthology.orchestra.run.smithi093.stdout: "max_completed_events": { 2023-12-14T18:36:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_completed_events", 2023-12-14T18:36:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "50", 2023-12-14T18:36:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.797 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "number of past completed events to remember", 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.798 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long the module is going to sleep", 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.799 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus", 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "cache": { 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cache", 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.800 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.801 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "exclude_perf_counters": { 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "exclude_perf_counters", 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.802 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.803 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.804 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.805 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.806 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.807 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.808 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.809 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools": { 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools", 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.810 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.811 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300", 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.812 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_interval": { 2023-12-14T18:36:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_interval", 2023-12-14T18:36:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.813 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15.0", 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-12-14T18:36:34.814 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.815 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "9283", 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.816 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-14T18:36:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "stale_cache_strategy": { 2023-12-14T18:36:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stale_cache_strategy", 2023-12-14T18:36:34.817 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "log", 2023-12-14T18:36:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.818 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "default", 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.819 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "default", 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "error" 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.820 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.821 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.822 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.823 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.824 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.825 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.826 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.827 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.828 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.829 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_snap_create": { 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_snap_create", 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.830 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "mirror_snapshot_schedule": { 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirror_snapshot_schedule", 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.831 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.832 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "trash_purge_schedule": { 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "trash_purge_schedule", 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.833 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "restful", 2023-12-14T18:36:34.834 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_auth": { 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_auth", 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.835 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.836 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.837 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.838 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.839 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.840 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.841 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.842 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.843 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.844 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.845 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.846 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.847 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.848 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.849 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.850 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.851 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.852 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.853 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rgw", 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.854 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.855 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.856 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.857 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.858 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.859 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.860 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.861 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rook", 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "drive_group_interval": { 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "drive_group_interval", 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300.0", 2023-12-14T18:36:34.862 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.863 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.864 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.865 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.866 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.867 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.868 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.869 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "storage_class": { 2023-12-14T18:36:34.870 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "storage_class", 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "local", 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.871 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "selftest", 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.872 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.873 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.874 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.875 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.876 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.877 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.878 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.879 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "roption1": { 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption1", 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.880 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "roption2": { 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption2", 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.881 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "xyz", 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.882 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption1": { 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption1", 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.883 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption2": { 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption2", 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.884 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption3": { 2023-12-14T18:36:34.885 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption3", 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.886 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption4": { 2023-12-14T18:36:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption4", 2023-12-14T18:36:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.887 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption5": { 2023-12-14T18:36:34.888 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption5", 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.889 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption6": { 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption6", 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.890 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption7": { 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption7", 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.891 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "42", 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.892 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "testkey": { 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testkey", 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.893 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "testlkey": { 2023-12-14T18:36:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testlkey", 2023-12-14T18:36:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.894 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.895 INFO:teuthology.orchestra.run.smithi093.stdout: "testnewline": { 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testnewline", 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.896 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.897 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_m_granularity": { 2023-12-14T18:36:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_m_granularity", 2023-12-14T18:36:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.898 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow minute scheduled snapshots", 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "dump_on_update": { 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dump_on_update", 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.899 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "dump database to debug log on update", 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.900 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.901 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.902 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.903 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.904 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.905 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.906 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.907 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stats", 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.908 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.909 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.910 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.911 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.912 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.913 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.914 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.915 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "status", 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.916 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.917 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.918 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.919 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.920 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.921 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.922 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.923 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telegraf", 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "address": { 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "address", 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.924 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-14T18:36:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.925 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15", 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.926 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.927 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.928 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.929 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.930 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.931 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.932 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.933 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.934 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telemetry", 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_basic": { 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_basic", 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.935 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_crash": { 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_crash", 2023-12-14T18:36:34.936 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-14T18:36:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.937 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_device": { 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_device", 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.938 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_ident": { 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_ident", 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.939 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.940 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_perf": { 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_perf", 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.941 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "contact": { 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "contact", 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.942 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "description": { 2023-12-14T18:36:34.943 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "description", 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.944 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "device_url": { 2023-12-14T18:36:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_url", 2023-12-14T18:36:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-14T18:36:34.945 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-12-14T18:36:34.946 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.947 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "24", 2023-12-14T18:36:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "8", 2023-12-14T18:36:34.948 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "last_opt_revision": { 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "last_opt_revision", 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.949 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1", 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.950 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard": { 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard", 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.951 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard_description": { 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard_description", 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.952 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.953 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.954 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.955 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.956 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.957 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.958 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.959 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.960 INFO:teuthology.orchestra.run.smithi093.stdout: "organization": { 2023-12-14T18:36:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "organization", 2023-12-14T18:36:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.961 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "proxy": { 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "proxy", 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.962 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "url": { 2023-12-14T18:36:34.963 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url", 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.964 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "test_orchestrator", 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.965 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.966 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.967 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.968 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.969 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.970 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.971 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.972 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.973 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.974 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.975 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.976 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.977 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.978 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.979 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.980 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_clones": { 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_clones", 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4", 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.981 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "snapshot_clone_delay": { 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snapshot_clone_delay", 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.982 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-14T18:36:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.983 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix", 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "discovery_interval": { 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "discovery_interval", 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:34.984 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "100", 2023-12-14T18:36:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.985 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "identifier": { 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "identifier", 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.986 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.987 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.988 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.989 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.990 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.991 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.992 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:34.993 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:34.994 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.995 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_host": { 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_host", 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.996 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_port": { 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_port", 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.997 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10051", 2023-12-14T18:36:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:34.998 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_sender": { 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_sender", 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:34.999 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.000 INFO:teuthology.orchestra.run.smithi093.stdout: "modules": [ 2023-12-14T18:36:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-12-14T18:36:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "dashboard", 2023-12-14T18:36:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "iostat", 2023-12-14T18:36:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "nfs", 2023-12-14T18:36:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "restful", 2023-12-14T18:36:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "snap_schedule" 2023-12-14T18:36:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: "available_modules": [ 2023-12-14T18:36:35.001 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alerts", 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.002 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How frequently to reexamine health status", 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.003 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.004 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.005 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.006 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.007 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.008 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.009 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.010 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_destination": { 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_destination", 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email address to send alerts to", 2023-12-14T18:36:35.011 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_from_name": { 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_from_name", 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "Ceph", 2023-12-14T18:36:35.012 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Email From: name", 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_host": { 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_host", 2023-12-14T18:36:35.013 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP server", 2023-12-14T18:36:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.014 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_password": { 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_password", 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.015 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Password to authenticate with", 2023-12-14T18:36:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_port": { 2023-12-14T18:36:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_port", 2023-12-14T18:36:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.016 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "465", 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP port", 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.017 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_sender": { 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_sender", 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.018 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SMTP envelope sender", 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_ssl": { 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_ssl", 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.019 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use SSL to connect to SMTP server", 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "smtp_user": { 2023-12-14T18:36:35.020 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "smtp_user", 2023-12-14T18:36:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.021 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "User to authenticate as", 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "balancer", 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.022 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "active": { 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "active", 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.023 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically balance PGs across cluster", 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_time": { 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_time", 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.024 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0000", 2023-12-14T18:36:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "beginning time of day to automatically balance", 2023-12-14T18:36:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:36:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.025 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "begin_weekday": { 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "begin_weekday", 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.026 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_max_iterations": { 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_max_iterations", 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.027 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "25", 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "250", 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum number of iterations to attempt optimization", 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_metrics": { 2023-12-14T18:36:35.028 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_metrics", 2023-12-14T18:36:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "pgs,objects,bytes", 2023-12-14T18:36:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "metrics with which to calculate OSD utilization", 2023-12-14T18:36:35.029 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "crush_compat_step": { 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crush_compat_step", 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.5", 2023-12-14T18:36:35.030 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0.001", 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "0.999", 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "aggressiveness of optimization", 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "end_time": { 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_time", 2023-12-14T18:36:35.031 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2359", 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ending time of day to automatically balance", 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "This is a time of day in the format HHMM.", 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.032 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "end_weekday": { 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "end_weekday", 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "0", 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "6", 2023-12-14T18:36:35.033 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.034 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.035 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.036 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.037 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.038 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.039 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.040 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "min_score": { 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_score", 2023-12-14T18:36:35.041 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "minimum score, below which no optimization is attempted", 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.042 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "upmap", 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.043 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "crush-compat", 2023-12-14T18:36:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "none", 2023-12-14T18:36:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap" 2023-12-14T18:36:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Balancer mode", 2023-12-14T18:36:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.044 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_ids": { 2023-12-14T18:36:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_ids", 2023-12-14T18:36:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.045 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "pools which the automatic balancing will be limited to", 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.046 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and attempt optimization", 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.047 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_deviation": { 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_deviation", 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "deviation below which no optimization is attempted", 2023-12-14T18:36:35.048 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2023-12-14T18:36:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "upmap_max_optimizations": { 2023-12-14T18:36:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "upmap_max_optimizations", 2023-12-14T18:36:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.049 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "maximum upmap optimizations to make per attempt", 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.050 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cephadm", 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_down_multiplier": { 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_down_multiplier", 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.051 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.052 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_refresh_rate": { 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_refresh_rate", 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "20", 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2023-12-14T18:36:35.053 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "agent_starting_port": { 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "agent_starting_port", 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4721", 2023-12-14T18:36:35.054 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_password": { 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_password", 2023-12-14T18:36:35.055 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web password", 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.056 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "alertmanager_web_user": { 2023-12-14T18:36:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "alertmanager_web_user", 2023-12-14T18:36:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.057 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Alertmanager web user", 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_ptrace": { 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_ptrace", 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.058 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.059 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_interval": { 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_interval", 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.060 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to autotune daemon memory", 2023-12-14T18:36:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "autotune_memory_target_ratio": { 2023-12-14T18:36:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "autotune_memory_target_ratio", 2023-12-14T18:36:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.061 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0.7", 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.062 INFO:teuthology.orchestra.run.smithi093.stdout: "cgroups_split": { 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cgroups_split", 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2023-12-14T18:36:35.063 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "config_checks_enabled": { 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_checks_enabled", 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.064 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "config_dashboard": { 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_dashboard", 2023-12-14T18:36:35.065 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2023-12-14T18:36:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.066 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_alertmanager": { 2023-12-14T18:36:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_alertmanager", 2023-12-14T18:36:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2023-12-14T18:36:35.067 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_base": { 2023-12-14T18:36:35.068 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_base", 2023-12-14T18:36:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph", 2023-12-14T18:36:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Container image name, without the tag", 2023-12-14T18:36:35.069 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_elasticsearch": { 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_elasticsearch", 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2023-12-14T18:36:35.070 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "elasticsearch container image", 2023-12-14T18:36:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_grafana": { 2023-12-14T18:36:35.071 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_grafana", 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/ceph-grafana:9.4.7", 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.072 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_haproxy": { 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_haproxy", 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.073 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "HAproxy container image", 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_agent": { 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_agent", 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.074 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger agent container image", 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.075 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_collector": { 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_collector", 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.076 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger collector container image", 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_jaeger_query": { 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_jaeger_query", 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.077 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2023-12-14T18:36:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Jaeger query container image", 2023-12-14T18:36:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.078 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_keepalived": { 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_keepalived", 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Keepalived container image", 2023-12-14T18:36:35.079 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_loki": { 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_loki", 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2023-12-14T18:36:35.080 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Loki container image", 2023-12-14T18:36:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_node_exporter": { 2023-12-14T18:36:35.081 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_node_exporter", 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.082 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_prometheus": { 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_prometheus", 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.083 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus container image", 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_promtail": { 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_promtail", 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.084 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2023-12-14T18:36:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Promtail container image", 2023-12-14T18:36:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.085 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "container_image_snmp_gateway": { 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_image_snmp_gateway", 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.086 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "SNMP Gateway container image", 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "container_init": { 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "container_init", 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.087 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Run podman/docker with `--init`", 2023-12-14T18:36:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.088 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "daemon_cache_timeout": { 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "daemon_cache_timeout", 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.089 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache service (daemon) inventory", 2023-12-14T18:36:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "default_cephadm_command_timeout": { 2023-12-14T18:36:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_cephadm_command_timeout", 2023-12-14T18:36:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.090 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "900", 2023-12-14T18:36:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2023-12-14T18:36:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.091 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "default_registry": { 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "default_registry", 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "docker.io", 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.092 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2023-12-14T18:36:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "device_cache_timeout": { 2023-12-14T18:36:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_cache_timeout", 2023-12-14T18:36:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.093 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1800", 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache device inventory", 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.094 INFO:teuthology.orchestra.run.smithi093.stdout: "device_enhanced_scan": { 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_enhanced_scan", 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use libstoragemgmt during device scans", 2023-12-14T18:36:35.095 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "facts_cache_timeout": { 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "facts_cache_timeout", 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:35.096 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "seconds to cache host facts data", 2023-12-14T18:36:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "host_check_interval": { 2023-12-14T18:36:35.097 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "host_check_interval", 2023-12-14T18:36:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to perform a host check", 2023-12-14T18:36:35.098 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.099 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.100 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "log_refresh_metadata": { 2023-12-14T18:36:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_refresh_metadata", 2023-12-14T18:36:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.101 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.102 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.103 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.104 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.105 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.106 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf": { 2023-12-14T18:36:35.107 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf", 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.108 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2023-12-14T18:36:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2023-12-14T18:36:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "*", 2023-12-14T18:36:35.109 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "max_count_per_host": { 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_count_per_host", 2023-12-14T18:36:35.110 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of daemons per service per host", 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.111 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "max_osd_draining_count": { 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_osd_draining_count", 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.112 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2023-12-14T18:36:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "migration_current": { 2023-12-14T18:36:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "migration_current", 2023-12-14T18:36:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.113 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "internal - do not modify", 2023-12-14T18:36:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.114 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "mode": { 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mode", 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "root", 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm-package", 2023-12-14T18:36:35.115 INFO:teuthology.orchestra.run.smithi093.stdout: "root" 2023-12-14T18:36:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "mode for remote execution of cephadm", 2023-12-14T18:36:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_alerts_path": { 2023-12-14T18:36:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_alerts_path", 2023-12-14T18:36:35.116 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2023-12-14T18:36:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "location of alerts to include in prometheus deployments", 2023-12-14T18:36:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.117 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_password": { 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_password", 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.118 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web password", 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "prometheus_web_user": { 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus_web_user", 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.119 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Prometheus web user", 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.120 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_insecure": { 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_insecure", 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.121 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_password": { 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_password", 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.122 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2023-12-14T18:36:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.123 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_url": { 2023-12-14T18:36:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_url", 2023-12-14T18:36:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.124 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "registry_username": { 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "registry_username", 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.125 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "secure_monitoring_stack": { 2023-12-14T18:36:35.126 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "secure_monitoring_stack", 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.127 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "service_discovery_port": { 2023-12-14T18:36:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "service_discovery_port", 2023-12-14T18:36:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8765", 2023-12-14T18:36:35.128 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "cephadm service discovery port", 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "ssh_config_file": { 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssh_config_file", 2023-12-14T18:36:35.129 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "customized SSH config file to connect to managed hosts", 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.130 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "use_agent": { 2023-12-14T18:36:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_agent", 2023-12-14T18:36:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.131 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "use_repo_digest": { 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "use_repo_digest", 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.132 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2023-12-14T18:36:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.133 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_failed_host_check": { 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_failed_host_check", 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.134 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if the host check fails", 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_daemons": { 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_daemons", 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.135 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2023-12-14T18:36:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.136 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_on_stray_hosts": { 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_on_stray_hosts", 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.137 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2023-12-14T18:36:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crash", 2023-12-14T18:36:35.138 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.139 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.140 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.141 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.142 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.143 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.144 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.145 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "retain_interval": { 2023-12-14T18:36:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retain_interval", 2023-12-14T18:36:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.146 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "31536000", 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain crashes before pruning them", 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_recent_interval": { 2023-12-14T18:36:35.147 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_recent_interval", 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1209600", 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "time interval in which to warn about recent crashes", 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.148 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dashboard", 2023-12-14T18:36:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.149 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.150 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_HOST": { 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_HOST", 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.151 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2023-12-14T18:36:35.152 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.153 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_ENABLED": { 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_ENABLED", 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.154 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "AUDIT_API_LOG_PAYLOAD": { 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.155 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.156 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "ENABLE_BROWSABLE_API": { 2023-12-14T18:36:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ENABLE_BROWSABLE_API", 2023-12-14T18:36:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.157 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_CEPHFS": { 2023-12-14T18:36:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2023-12-14T18:36:35.158 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.159 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.160 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_ISCSI": { 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.161 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.162 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_MIRRORING": { 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.163 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_NFS": { 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_NFS", 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.164 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RBD": { 2023-12-14T18:36:35.165 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RBD", 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.166 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "FEATURE_TOGGLE_RGW": { 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "FEATURE_TOGGLE_RGW", 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.167 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.168 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.169 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_PASSWORD": { 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_PASSWORD", 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.170 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_SSL_VERIFY": { 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.171 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.172 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_URL": { 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_URL", 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.173 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_API_USERNAME": { 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_API_USERNAME", 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:35.174 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_FRONTEND_API_URL": { 2023-12-14T18:36:35.175 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.176 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.177 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "ISCSI_API_SSL_VERIFICATION": { 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.178 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.179 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "ISSUE_TRACKER_API_KEY": { 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ISSUE_TRACKER_API_KEY", 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.180 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_HOST": { 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_HOST", 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.181 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.182 INFO:teuthology.orchestra.run.smithi093.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.183 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.184 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2023-12-14T18:36:35.185 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.186 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2023-12-14T18:36:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2023-12-14T18:36:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.187 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2023-12-14T18:36:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2023-12-14T18:36:35.188 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.189 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2023-12-14T18:36:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2023-12-14T18:36:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.190 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.191 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.192 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.193 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_ENABLED": { 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_ENABLED", 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.194 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.195 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2023-12-14T18:36:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2023-12-14T18:36:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2023-12-14T18:36:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.196 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.197 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.198 INFO:teuthology.orchestra.run.smithi093.stdout: "PWD_POLICY_MIN_LENGTH": { 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8", 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.199 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "REST_REQUESTS_TIMEOUT": { 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "REST_REQUESTS_TIMEOUT", 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "45", 2023-12-14T18:36:35.200 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ACCESS_KEY": { 2023-12-14T18:36:35.201 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ACCESS_KEY", 2023-12-14T18:36:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.202 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_ADMIN_RESOURCE": { 2023-12-14T18:36:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2023-12-14T18:36:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.203 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "admin", 2023-12-14T18:36:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.204 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SECRET_KEY": { 2023-12-14T18:36:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SECRET_KEY", 2023-12-14T18:36:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.205 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "RGW_API_SSL_VERIFY": { 2023-12-14T18:36:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "RGW_API_SSL_VERIFY", 2023-12-14T18:36:35.206 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.207 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_SPAN": { 2023-12-14T18:36:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2023-12-14T18:36:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:35.208 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.209 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2023-12-14T18:36:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2023-12-14T18:36:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.210 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2023-12-14T18:36:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2023-12-14T18:36:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.211 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.212 INFO:teuthology.orchestra.run.smithi093.stdout: "cross_origin_url": { 2023-12-14T18:36:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cross_origin_url", 2023-12-14T18:36:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.213 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "crt_file": { 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "crt_file", 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.214 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.215 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "debug": { 2023-12-14T18:36:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "debug", 2023-12-14T18:36:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.216 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Enable/disable debug options", 2023-12-14T18:36:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "jwt_token_ttl": { 2023-12-14T18:36:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "jwt_token_ttl", 2023-12-14T18:36:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.217 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "28800", 2023-12-14T18:36:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.218 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-12-14T18:36:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-12-14T18:36:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.219 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.220 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.221 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.222 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.223 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.224 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.225 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.226 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.227 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "motd": { 2023-12-14T18:36:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "motd", 2023-12-14T18:36:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.228 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "The message of the day", 2023-12-14T18:36:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect_resolve_ip_addr": { 2023-12-14T18:36:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "redirect_resolve_ip_addr", 2023-12-14T18:36:35.229 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.230 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.231 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-12-14T18:36:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-12-14T18:36:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.232 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8080", 2023-12-14T18:36:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.233 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.234 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl_server_port": { 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl_server_port", 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.235 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8443", 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.236 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-12-14T18:36:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-12-14T18:36:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "redirect", 2023-12-14T18:36:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.237 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "redirect" 2023-12-14T18:36:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-12-14T18:36:35.238 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.239 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "url_prefix": { 2023-12-14T18:36:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url_prefix", 2023-12-14T18:36:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.240 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.241 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-12-14T18:36:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_monitoring": { 2023-12-14T18:36:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_monitoring", 2023-12-14T18:36:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.242 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "monitor device health metrics", 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.243 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.244 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.245 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.246 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.247 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.248 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.249 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.250 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "mark_out_threshold": { 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mark_out_threshold", 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.251 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "2419200", 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "automatically mark OSD if it may fail before this long", 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.252 INFO:teuthology.orchestra.run.smithi093.stdout: "pool_name": { 2023-12-14T18:36:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pool_name", 2023-12-14T18:36:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "device_health_metrics", 2023-12-14T18:36:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.253 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name of pool in which to store device health metrics", 2023-12-14T18:36:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "retention_period": { 2023-12-14T18:36:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "retention_period", 2023-12-14T18:36:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.254 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15552000", 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long to retain device health metrics", 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.255 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_frequency": { 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_frequency", 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to scrape device health metrics", 2023-12-14T18:36:35.256 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "self_heal": { 2023-12-14T18:36:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "self_heal", 2023-12-14T18:36:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.257 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "preemptively heal cluster around devices that may fail", 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:35.258 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how frequently to wake up and check device health", 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.259 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "warn_threshold": { 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "warn_threshold", 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7257600", 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.260 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "raise health warning if OSD may fail before this long", 2023-12-14T18:36:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.261 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "diskprediction_local", 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.262 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.263 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.264 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.265 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.266 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.267 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.268 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "predict_interval": { 2023-12-14T18:36:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predict_interval", 2023-12-14T18:36:35.269 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "86400", 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.270 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "predictor_model": { 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "predictor_model", 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "prophetstor", 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.271 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.272 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "600", 2023-12-14T18:36:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.273 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "influx", 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": false, 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "influxdb python module not found", 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "batch_size": { 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "batch_size", 2023-12-14T18:36:35.274 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5000", 2023-12-14T18:36:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2023-12-14T18:36:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.275 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "database": { 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "database", 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "ceph", 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.276 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2023-12-14T18:36:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "hostname": { 2023-12-14T18:36:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "hostname", 2023-12-14T18:36:35.277 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server hostname", 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.278 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "30", 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "5", 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.279 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.280 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.281 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.282 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.283 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.284 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.285 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.286 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "password": { 2023-12-14T18:36:35.287 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "password", 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "password of InfluxDB server user", 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.288 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "port": { 2023-12-14T18:36:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "port", 2023-12-14T18:36:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "8086", 2023-12-14T18:36:35.289 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "InfluxDB server port", 2023-12-14T18:36:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.290 INFO:teuthology.orchestra.run.smithi093.stdout: "ssl": { 2023-12-14T18:36:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ssl", 2023-12-14T18:36:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "false", 2023-12-14T18:36:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.291 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "threads": { 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threads", 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.292 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "32", 2023-12-14T18:36:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2023-12-14T18:36:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.293 INFO:teuthology.orchestra.run.smithi093.stdout: "username": { 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "username", 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "username of InfluxDB server user", 2023-12-14T18:36:35.294 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "verify_ssl": { 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "verify_ssl", 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "true", 2023-12-14T18:36:35.295 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.296 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "insights", 2023-12-14T18:36:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.297 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.298 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.299 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.300 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.301 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.302 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.303 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.304 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "iostat", 2023-12-14T18:36:35.305 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.305 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.305 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.305 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.305 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.305 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.305 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.305 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.305 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.306 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.307 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.308 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.308 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.308 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.308 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.308 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.308 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.308 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.308 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.308 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.309 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.310 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.311 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "k8sevents", 2023-12-14T18:36:35.312 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.313 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.313 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.313 INFO:teuthology.orchestra.run.smithi093.stdout: "ceph_event_retention_days": { 2023-12-14T18:36:35.313 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "ceph_event_retention_days", 2023-12-14T18:36:35.313 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.313 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.313 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.313 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "7", 2023-12-14T18:36:35.313 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.314 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.314 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.314 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Days to hold ceph event information within local cache", 2023-12-14T18:36:35.314 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.314 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.314 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.314 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.314 INFO:teuthology.orchestra.run.smithi093.stdout: "config_check_secs": { 2023-12-14T18:36:35.314 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "config_check_secs", 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "10", 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.315 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.316 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.317 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.317 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.317 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.317 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.317 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.317 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.317 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.317 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.317 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.318 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.319 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.320 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.321 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.322 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "localpool", 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.323 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "failure_domain": { 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "failure_domain", 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "host", 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.324 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "failure domain for any created local pool", 2023-12-14T18:36:35.325 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2023-12-14T18:36:35.325 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.325 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.325 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.325 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.325 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.325 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.325 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.325 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.326 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.327 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.328 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.329 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.329 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.329 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.329 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.329 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.329 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.329 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.329 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.329 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.330 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.331 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.332 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.332 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.332 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.332 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.332 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.332 INFO:teuthology.orchestra.run.smithi093.stdout: "min_size": { 2023-12-14T18:36:35.332 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "min_size", 2023-12-14T18:36:35.332 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.332 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default min_size for any created local pool", 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.333 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "num_rep": { 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "num_rep", 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3", 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.334 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default replica count for any created local pool", 2023-12-14T18:36:35.335 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.335 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.335 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.335 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.335 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_num": { 2023-12-14T18:36:35.335 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_num", 2023-12-14T18:36:35.335 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.335 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.335 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.336 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "128", 2023-12-14T18:36:35.336 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.336 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.336 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.336 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "default pg_num for any created local pool", 2023-12-14T18:36:35.336 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.336 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.336 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.336 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.337 INFO:teuthology.orchestra.run.smithi093.stdout: "prefix": { 2023-12-14T18:36:35.337 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prefix", 2023-12-14T18:36:35.337 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.337 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.337 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.337 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.337 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.337 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.337 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.338 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "name prefix for any created local pool", 2023-12-14T18:36:35.338 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.338 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.338 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.338 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.338 INFO:teuthology.orchestra.run.smithi093.stdout: "subtree": { 2023-12-14T18:36:35.338 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "subtree", 2023-12-14T18:36:35.338 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.338 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "rack", 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "CRUSH level for which to create a local pool", 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.339 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mds_autoscaler", 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.340 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.341 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.342 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.343 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.344 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.344 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.344 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.344 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.344 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.344 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.344 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.344 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.344 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.345 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.346 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.347 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.347 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.347 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.347 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.347 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.347 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.347 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.347 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.347 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirroring", 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.348 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.349 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.349 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.349 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.349 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.349 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.349 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.349 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.349 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.349 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.350 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.351 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.352 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.353 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.354 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.355 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "nfs", 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.356 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.357 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.357 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.357 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.357 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.357 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.357 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.357 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.357 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.357 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.358 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.359 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.360 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.361 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.362 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-12-14T18:36:35.363 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "fail_fs": { 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "fail_fs", 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.364 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.365 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.366 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.367 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.368 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.368 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.368 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.368 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.368 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.368 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.368 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.368 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.368 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.369 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.370 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.371 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator": { 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "orchestrator", 2023-12-14T18:36:35.372 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: "cephadm", 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: "rook", 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: "test_orchestrator" 2023-12-14T18:36:35.373 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Orchestrator backend", 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_perf_query", 2023-12-14T18:36:35.374 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.375 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.376 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.376 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.376 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.376 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.376 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.376 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.376 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.376 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.376 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.377 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.378 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.378 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.378 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.378 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.378 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.378 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.378 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.378 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.378 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.379 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.380 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.381 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.381 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.381 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.381 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.381 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.381 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.381 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.381 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.381 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "osd_support", 2023-12-14T18:36:35.382 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.383 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.384 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.385 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.386 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.387 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.387 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.387 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.387 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.387 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.387 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.387 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.387 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.387 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.388 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.389 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.389 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.389 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.389 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.389 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.389 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.389 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.389 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.389 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "pg_autoscaler", 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.390 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.391 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.392 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.393 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.393 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.393 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.393 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.393 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.393 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.393 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.393 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.393 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.394 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.395 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.396 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.397 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.398 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.398 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.398 INFO:teuthology.orchestra.run.smithi093.stdout: "noautoscale": { 2023-12-14T18:36:35.398 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "noautoscale", 2023-12-14T18:36:35.398 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.398 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.398 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.398 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.398 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "global autoscale flag", 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Option to turn on/off the autoscaler for all pools", 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:35.399 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.400 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: "threshold": { 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "threshold", 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "3.0", 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1.0", 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.401 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.402 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "scaling threshold", 2023-12-14T18:36:35.402 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2023-12-14T18:36:35.402 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.402 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.402 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.402 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.402 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.402 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.402 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "progress", 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_pg_recovery_event": { 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_pg_recovery_event", 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.403 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow the module to show pg recovery progress", 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-12-14T18:36:35.404 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.405 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.406 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.407 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.408 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.409 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.410 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.411 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.412 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.413 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.413 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.413 INFO:teuthology.orchestra.run.smithi093.stdout: "max_completed_events": { 2023-12-14T18:36:35.413 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_completed_events", 2023-12-14T18:36:35.413 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.413 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.413 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.413 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "50", 2023-12-14T18:36:35.413 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "number of past completed events to remember", 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: "sleep_interval": { 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "sleep_interval", 2023-12-14T18:36:35.414 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "5", 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "how long the module is going to sleep", 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.415 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "prometheus", 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: "cache": { 2023-12-14T18:36:35.416 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "cache", 2023-12-14T18:36:35.417 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.417 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.417 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.417 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.417 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.417 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.417 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.417 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.417 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: "exclude_perf_counters": { 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "exclude_perf_counters", 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.418 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.419 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.419 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.419 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Do not include perf-counters in the metrics output", 2023-12-14T18:36:35.419 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2023-12-14T18:36:35.419 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.419 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.419 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.419 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.419 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.420 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.421 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.422 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.423 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.424 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.425 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.426 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.426 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.426 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.426 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.426 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.426 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.426 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.426 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.426 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools": { 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools", 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.427 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_stats_pools_refresh_interval": { 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_stats_pools_refresh_interval", 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300", 2023-12-14T18:36:35.428 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.429 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.429 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.429 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.429 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.429 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.429 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.429 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.429 INFO:teuthology.orchestra.run.smithi093.stdout: "scrape_interval": { 2023-12-14T18:36:35.429 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "scrape_interval", 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15.0", 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.430 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "::", 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.431 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.432 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.432 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2023-12-14T18:36:35.432 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.432 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.432 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.432 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.432 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-12-14T18:36:35.432 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-12-14T18:36:35.432 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.433 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.433 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.433 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "9283", 2023-12-14T18:36:35.433 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.433 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.433 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.433 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "the port on which the module listens for HTTP requests", 2023-12-14T18:36:35.433 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.433 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.434 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.434 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.434 INFO:teuthology.orchestra.run.smithi093.stdout: "stale_cache_strategy": { 2023-12-14T18:36:35.434 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stale_cache_strategy", 2023-12-14T18:36:35.434 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.434 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.434 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.434 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "log", 2023-12-14T18:36:35.434 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_behaviour": { 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_behaviour", 2023-12-14T18:36:35.435 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "default", 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: "default", 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: "error" 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.436 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.437 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.437 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.437 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.437 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.437 INFO:teuthology.orchestra.run.smithi093.stdout: "standby_error_status_code": { 2023-12-14T18:36:35.437 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "standby_error_status_code", 2023-12-14T18:36:35.437 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.437 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.437 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "500", 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "400", 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "599", 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.438 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.439 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.440 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.441 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.441 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.441 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.441 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.441 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.441 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.441 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.441 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.441 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.442 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.443 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.444 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.445 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_snap_create": { 2023-12-14T18:36:35.446 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_snap_create", 2023-12-14T18:36:35.447 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.447 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.447 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.447 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10", 2023-12-14T18:36:35.447 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.447 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.447 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.447 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.447 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: "mirror_snapshot_schedule": { 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "mirror_snapshot_schedule", 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.448 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: "trash_purge_schedule": { 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "trash_purge_schedule", 2023-12-14T18:36:35.449 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.450 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.451 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.451 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.451 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.451 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.451 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "restful", 2023-12-14T18:36:35.451 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.451 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.451 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.451 INFO:teuthology.orchestra.run.smithi093.stdout: "enable_auth": { 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enable_auth", 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.452 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: "key_file": { 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "key_file", 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.453 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.454 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.455 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.456 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.457 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.457 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.457 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.457 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.457 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.457 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.457 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.457 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.457 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.458 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.459 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.460 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.461 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.461 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.461 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.461 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.461 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.461 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.461 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.461 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.461 INFO:teuthology.orchestra.run.smithi093.stdout: "server_addr": { 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_addr", 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.462 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: "server_port": { 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "server_port", 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.463 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.464 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rgw", 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.465 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.466 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.467 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.468 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.468 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.468 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.468 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.468 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.468 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.468 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.468 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.468 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.469 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.470 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.471 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.472 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.472 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.472 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.472 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.472 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.472 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.472 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.472 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.472 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rook", 2023-12-14T18:36:35.473 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.473 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.473 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.473 INFO:teuthology.orchestra.run.smithi093.stdout: "drive_group_interval": { 2023-12-14T18:36:35.473 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "drive_group_interval", 2023-12-14T18:36:35.473 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:35.473 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.473 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.473 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "300.0", 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "interval in seconds between re-application of applied drive_groups", 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.474 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.475 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.476 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.477 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.477 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.477 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.477 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.477 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.477 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.477 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.477 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.477 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.478 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.479 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.480 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.481 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.481 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.481 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.481 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.481 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.481 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.481 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.481 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.481 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "storage_class": { 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "storage_class", 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "local", 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.482 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "storage class name for LSO-discovered PVs", 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "selftest", 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.483 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.484 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.485 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.485 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.485 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.485 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.485 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.485 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.485 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.485 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.485 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.486 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.487 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.488 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.488 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.488 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.488 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.488 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.488 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.488 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.488 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.488 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.489 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.490 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: "roption1": { 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption1", 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.491 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: "roption2": { 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "roption2", 2023-12-14T18:36:35.492 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "xyz", 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.493 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.494 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.494 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption1": { 2023-12-14T18:36:35.494 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption1", 2023-12-14T18:36:35.494 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.494 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.494 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.494 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.494 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.494 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption2": { 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption2", 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.495 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.496 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption3": { 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption3", 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "float", 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.497 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption4": { 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption4", 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.498 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption5": { 2023-12-14T18:36:35.499 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption5", 2023-12-14T18:36:35.500 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.500 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.500 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.500 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.500 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.500 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.500 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.500 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.500 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption6": { 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption6", 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.501 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: "rwoption7": { 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rwoption7", 2023-12-14T18:36:35.502 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "1", 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "42", 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.503 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: "testkey": { 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testkey", 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.504 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.505 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.505 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.505 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.505 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.505 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.505 INFO:teuthology.orchestra.run.smithi093.stdout: "testlkey": { 2023-12-14T18:36:35.505 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testlkey", 2023-12-14T18:36:35.505 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.505 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.506 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "testnewline": { 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "testnewline", 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.507 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.508 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.508 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.508 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.508 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.508 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.508 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.508 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.508 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-12-14T18:36:35.508 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "allow_m_granularity": { 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "allow_m_granularity", 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.509 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "allow minute scheduled snapshots", 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: "dump_on_update": { 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "dump_on_update", 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.510 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "dump database to debug log on update", 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.511 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.512 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.513 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.513 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.513 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.513 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.513 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.513 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.513 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.513 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.513 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.514 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.515 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.516 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.517 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.518 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.518 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.518 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.518 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.518 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.518 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.518 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.518 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.518 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "stats", 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.519 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.520 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.521 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.521 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.521 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.521 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.521 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.521 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.521 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.521 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.521 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.522 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.523 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.523 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.523 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.523 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.523 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.523 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.523 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.523 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.523 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.524 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.524 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.524 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.524 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.524 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.524 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.524 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.524 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.524 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.525 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.525 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.525 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.525 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.525 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.525 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.525 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.525 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.525 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.526 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "status", 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.527 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.528 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.528 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.528 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.528 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.528 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.528 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.528 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.528 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.528 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.529 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.530 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.531 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.532 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.533 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.534 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.534 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.534 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.534 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.534 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.534 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.534 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.534 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.534 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telegraf", 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: "address": { 2023-12-14T18:36:35.535 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "address", 2023-12-14T18:36:35.536 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.536 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.536 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.536 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2023-12-14T18:36:35.536 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.536 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.536 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.536 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.536 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "15", 2023-12-14T18:36:35.537 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.538 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.539 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.540 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.541 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.541 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.541 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.541 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.541 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.541 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.541 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.541 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.541 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.542 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.543 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.544 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.545 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.546 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.546 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "telemetry", 2023-12-14T18:36:35.546 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.546 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.546 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.546 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_basic": { 2023-12-14T18:36:35.546 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_basic", 2023-12-14T18:36:35.546 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.546 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share basic cluster information (size, version)", 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.547 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.548 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_crash": { 2023-12-14T18:36:35.548 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_crash", 2023-12-14T18:36:35.548 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.548 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.548 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.548 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.548 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.548 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.548 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_device": { 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_device", 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.549 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "True", 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.550 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_ident": { 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_ident", 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2023-12-14T18:36:35.551 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: "channel_perf": { 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "channel_perf", 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.552 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.553 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.553 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.553 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Share various performance metrics of a cluster", 2023-12-14T18:36:35.553 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.553 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.553 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.553 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.553 INFO:teuthology.orchestra.run.smithi093.stdout: "contact": { 2023-12-14T18:36:35.553 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "contact", 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.554 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: "description": { 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "description", 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.555 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: "device_url": { 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "device_url", 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.556 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.557 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.557 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/device", 2023-12-14T18:36:35.557 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.557 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.557 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.557 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.557 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.557 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.557 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: "enabled": { 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "enabled", 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.558 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.559 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "24", 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "8", 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.560 INFO:teuthology.orchestra.run.smithi093.stdout: "last_opt_revision": { 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "last_opt_revision", 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "1", 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.561 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard": { 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard", 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.562 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: "leaderboard_description": { 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "leaderboard_description", 2023-12-14T18:36:35.563 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.564 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.565 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.565 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.565 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.565 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.565 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.565 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.565 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.565 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.565 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.566 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.567 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.568 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.569 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.569 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.569 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.569 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.569 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.569 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.569 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.569 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.569 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.570 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.571 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: "organization": { 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "organization", 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.572 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: "proxy": { 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "proxy", 2023-12-14T18:36:35.573 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.574 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: "url": { 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "url", 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "https://telemetry.ceph.com/report", 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.575 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "test_orchestrator", 2023-12-14T18:36:35.576 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.577 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.578 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.579 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.579 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.579 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.579 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.579 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.579 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.579 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.579 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.579 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.580 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.581 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.582 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.583 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.584 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.585 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.585 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.585 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.585 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.585 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.585 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.585 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.585 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.585 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.586 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.587 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.588 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.588 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.588 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.588 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.588 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.588 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.588 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.588 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.588 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.589 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.590 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.591 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: "max_concurrent_clones": { 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "max_concurrent_clones", 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "4", 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.592 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.593 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.593 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Number of asynchronous cloner threads", 2023-12-14T18:36:35.593 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.593 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.593 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.593 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.593 INFO:teuthology.orchestra.run.smithi093.stdout: "snapshot_clone_delay": { 2023-12-14T18:36:35.593 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snapshot_clone_delay", 2023-12-14T18:36:35.593 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "0", 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.594 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix", 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: "can_run": true, 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: "error_string": "", 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: "module_options": { 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: "discovery_interval": { 2023-12-14T18:36:35.595 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "discovery_interval", 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "uint", 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "100", 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.596 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.597 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.597 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.597 INFO:teuthology.orchestra.run.smithi093.stdout: "identifier": { 2023-12-14T18:36:35.597 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "identifier", 2023-12-14T18:36:35.597 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.597 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.597 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.597 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.597 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: "interval": { 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "interval", 2023-12-14T18:36:35.598 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "secs", 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "60", 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.599 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: "log_level": { 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_level", 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.600 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.601 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.601 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.601 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.601 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.601 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.601 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.601 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.601 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.601 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster": { 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster", 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.602 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_cluster_level": { 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_cluster_level", 2023-12-14T18:36:35.603 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "info", 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [ 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "", 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "critical", 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "debug", 2023-12-14T18:36:35.604 INFO:teuthology.orchestra.run.smithi093.stdout: "error", 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: "info", 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: "warning" 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: "log_to_file": { 2023-12-14T18:36:35.605 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "log_to_file", 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "bool", 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 1, 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "False", 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.606 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_host": { 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_host", 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "", 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.607 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.608 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.608 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.608 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.608 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.608 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.608 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.608 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_port": { 2023-12-14T18:36:35.608 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_port", 2023-12-14T18:36:35.608 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "int", 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "10051", 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.609 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: "zabbix_sender": { 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "zabbix_sender", 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "str", 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: "level": "advanced", 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: "flags": 0, 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: "default_value": "/usr/bin/zabbix_sender", 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: "min": "", 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: "max": "", 2023-12-14T18:36:35.610 INFO:teuthology.orchestra.run.smithi093.stdout: "enum_allowed": [], 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: "desc": "", 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: "long_desc": "", 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: "tags": [], 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: "see_also": [] 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: "services": { 2023-12-14T18:36:35.611 INFO:teuthology.orchestra.run.smithi093.stdout: "dashboard": "https://172.21.15.93:8443/" 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: "always_on_modules": { 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: "octopus": [ 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-12-14T18:36:35.612 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: "pacific": [ 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-12-14T18:36:35.613 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: "quincy": [ 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-12-14T18:36:35.614 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-12-14T18:36:35.615 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-12-14T18:36:35.615 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-12-14T18:36:35.615 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-12-14T18:36:35.615 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-12-14T18:36:35.615 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-12-14T18:36:35.615 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-12-14T18:36:35.615 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-12-14T18:36:35.615 INFO:teuthology.orchestra.run.smithi093.stdout: ], 2023-12-14T18:36:35.615 INFO:teuthology.orchestra.run.smithi093.stdout: "reef": [ 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "balancer", 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "crash", 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "devicehealth", 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "orchestrator", 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "pg_autoscaler", 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "progress", 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "rbd_support", 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "status", 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "telemetry", 2023-12-14T18:36:35.616 INFO:teuthology.orchestra.run.smithi093.stdout: "volumes" 2023-12-14T18:36:35.617 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:35.617 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.617 INFO:teuthology.orchestra.run.smithi093.stdout: "last_failure_osd_epoch": 87, 2023-12-14T18:36:35.617 INFO:teuthology.orchestra.run.smithi093.stdout: "active_clients": [ 2023-12-14T18:36:35.617 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.617 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "devicehealth", 2023-12-14T18:36:35.617 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-12-14T18:36:35.617 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.617 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-12-14T18:36:35.618 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-12-14T18:36:35.618 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 2474777880 2023-12-14T18:36:35.618 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.618 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:35.618 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.618 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.618 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "libcephsqlite", 2023-12-14T18:36:35.618 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-12-14T18:36:35.618 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.619 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-12-14T18:36:35.619 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-12-14T18:36:35.619 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3388291355 2023-12-14T18:36:35.619 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.619 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:35.619 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.619 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.619 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "rbd_support", 2023-12-14T18:36:35.619 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 1556392370 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-12-14T18:36:35.620 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-12-14T18:36:35.621 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.621 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-12-14T18:36:35.621 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-12-14T18:36:35.621 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 879896704 2023-12-14T18:36:35.621 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.621 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:35.621 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.621 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.621 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "snap_schedule", 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 1654039817 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: "name": "volumes", 2023-12-14T18:36:35.622 INFO:teuthology.orchestra.run.smithi093.stdout: "addrvec": [ 2023-12-14T18:36:35.623 INFO:teuthology.orchestra.run.smithi093.stdout: { 2023-12-14T18:36:35.623 INFO:teuthology.orchestra.run.smithi093.stdout: "type": "v2", 2023-12-14T18:36:35.623 INFO:teuthology.orchestra.run.smithi093.stdout: "addr": "172.21.15.93:0", 2023-12-14T18:36:35.623 INFO:teuthology.orchestra.run.smithi093.stdout: "nonce": 3860602011 2023-12-14T18:36:35.623 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.623 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:35.623 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:35.623 INFO:teuthology.orchestra.run.smithi093.stdout: ] 2023-12-14T18:36:35.623 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:36:35.625 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3521825190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:36:35.625 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/16513905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:36:35.625 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:33 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/986084814' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-14T18:36:35.625 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:34 smithi097 ceph-mon[43486]: pgmap v314: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:35.625 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3521825190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:36:35.626 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/16513905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:36:35.626 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:33 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/986084814' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-14T18:36:35.626 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:34 smithi160 ceph-mon[43840]: pgmap v314: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:35.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56a80a12c0 msgr2=0x7f56a80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:35.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56a80a12c0 0x7f56a80a3770 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f56ac009be0 tx=0x7f56ac009260 comp rx=0 tx=0).stop 2023-12-14T18:36:35.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81051b0 msgr2=0x7f56b819ad20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:35.626 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81051b0 0x7f56b819ad20 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f56b0007ee0 tx=0x7f56b000d3b0 comp rx=0 tx=0).stop 2023-12-14T18:36:35.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 shutdown_connections 2023-12-14T18:36:35.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f56a80a12c0 0x7f56a80a3770 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:35.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f56b81051b0 0x7f56b819ad20 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:35.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f56b81009e0 0x7f56b8196380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:35.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 --2- 172.21.15.93:0/986084814 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f56b8101390 0x7f56b81968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:35.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 >> 172.21.15.93:0/986084814 conn(0x7f56b8077520 msgr2=0x7f56b80ff720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:35.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 shutdown_connections 2023-12-14T18:36:35.627 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:33.750+0000 7f56befca700 1 -- 172.21.15.93:0/986084814 wait complete. 2023-12-14T18:36:35.632 DEBUG:tasks.check_counter:Getting stats from a 2023-12-14T18:36:35.633 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.a perf dump 2023-12-14T18:36:35.633 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3521825190' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:36:35.633 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/16513905' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:36:35.633 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:33 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/986084814' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2023-12-14T18:36:35.633 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:34 smithi093 ceph-mon[42070]: pgmap v314: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:36.111 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:36:37.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:36 smithi160 ceph-mon[43840]: pgmap v315: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:37.267 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:36 smithi093 ceph-mon[42070]: pgmap v315: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:37.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:36 smithi097 ceph-mon[43486]: pgmap v315: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:38.466 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:36:38.466 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:38.466 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 0, 2023-12-14T18:36:38.467 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2, 2023-12-14T18:36:38.467 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 0, 2023-12-14T18:36:38.467 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 0, 2023-12-14T18:36:38.467 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-12-14T18:36:38.467 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-12-14T18:36:38.467 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.001383016, 2023-12-14T18:36:38.468 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000000000, 2023-12-14T18:36:38.468 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.000293306, 2023-12-14T18:36:38.468 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:38.468 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:38.468 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:38.468 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:38.469 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:38.469 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.469 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:38.469 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:38.469 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:38.469 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:38.470 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.470 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:36:38.470 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:36:38.470 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.470 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:38.470 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 211, 2023-12-14T18:36:38.470 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 199, 2023-12-14T18:36:38.471 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 1420624, 2023-12-14T18:36:38.471 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 112496, 2023-12-14T18:36:38.471 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-12-14T18:36:38.471 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-12-14T18:36:38.471 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.034832778, 2023-12-14T18:36:38.471 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.011259213, 2023-12-14T18:36:38.471 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.026698541, 2023-12-14T18:36:38.472 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.004013999, 2023-12-14T18:36:38.472 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:38.472 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 199, 2023-12-14T18:36:38.472 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.012300714, 2023-12-14T18:36:38.472 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000061812 2023-12-14T18:36:38.472 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.473 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:38.473 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:38.473 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:38.473 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:38.473 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.473 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 1420624, 2023-12-14T18:36:38.473 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 112496 2023-12-14T18:36:38.474 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.474 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:38.474 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 2, 2023-12-14T18:36:38.474 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 150, 2023-12-14T18:36:38.474 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 320, 2023-12-14T18:36:38.474 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 178128, 2023-12-14T18:36:38.475 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 5, 2023-12-14T18:36:38.475 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 3, 2023-12-14T18:36:38.475 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.012197717, 2023-12-14T18:36:38.475 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.007977019, 2023-12-14T18:36:38.475 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.001943557, 2023-12-14T18:36:38.475 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:38.475 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:38.476 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 147, 2023-12-14T18:36:38.476 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.017592049, 2023-12-14T18:36:38.476 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000119673 2023-12-14T18:36:38.476 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.476 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:38.476 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:38.476 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:38.477 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:38.477 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.477 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:36:38.477 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 178128 2023-12-14T18:36:38.477 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.477 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-12-14T18:36:38.478 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-12-14T18:36:38.478 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-12-14T18:36:38.478 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.478 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-12-14T18:36:38.478 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:38.478 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:38.478 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:38.479 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:38.479 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:38.479 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:38.479 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:38.479 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:38.479 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:38.479 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:38.480 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:38.480 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:38.480 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:38.480 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:38.480 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:38.480 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:38.480 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:38.481 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:38.481 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:38.481 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:38.481 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:38.481 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:38.481 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:38.481 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:38.482 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:38.482 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:38.482 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:38.482 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:38.482 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:38.482 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:38.482 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:38.483 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-12-14T18:36:38.483 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:38.483 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:38.483 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:38.483 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:38.483 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 149057, 2023-12-14T18:36:38.483 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 32, 2023-12-14T18:36:38.484 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:38.484 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:38.484 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-12-14T18:36:38.484 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-12-14T18:36:38.484 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:38.484 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:38.484 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:38.485 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:38.485 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-12-14T18:36:38.485 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-12-14T18:36:38.485 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:38.485 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:38.485 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:38.486 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-12-14T18:36:38.486 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-12-14T18:36:38.486 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-12-14T18:36:38.486 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:38.486 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-12-14T18:36:38.486 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:38.486 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-12-14T18:36:38.487 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:38.487 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:38.487 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-12-14T18:36:38.487 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-12-14T18:36:38.487 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-12-14T18:36:38.487 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 209, 2023-12-14T18:36:38.487 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 1388427, 2023-12-14T18:36:38.488 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:38.488 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 209, 2023-12-14T18:36:38.488 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-12-14T18:36:38.488 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-12-14T18:36:38.488 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 209, 2023-12-14T18:36:38.488 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1388427, 2023-12-14T18:36:38.488 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-12-14T18:36:38.489 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:38.489 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:38.489 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:38.489 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:38.489 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:38.489 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:36:38.778 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.a 2023-12-14T18:36:38.779 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.a 2023-12-14T18:36:38.779 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.a 2023-12-14T18:36:38.779 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.a 2023-12-14T18:36:38.779 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2023-12-14T18:36:38.779 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2023-12-14T18:36:38.779 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2023-12-14T18:36:38.779 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.a 2023-12-14T18:36:38.780 DEBUG:tasks.check_counter:Getting stats from d 2023-12-14T18:36:38.780 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.d perf dump 2023-12-14T18:36:39.065 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:39 smithi093 ceph-mon[42070]: pgmap v316: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:39.093 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:36:39.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:39 smithi097 ceph-mon[43486]: pgmap v316: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:39.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:39 smithi160 ceph-mon[43840]: pgmap v316: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:40.053 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:36:40.054 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:40.054 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 0, 2023-12-14T18:36:40.054 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2, 2023-12-14T18:36:40.054 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 0, 2023-12-14T18:36:40.054 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 0, 2023-12-14T18:36:40.054 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-12-14T18:36:40.055 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-12-14T18:36:40.055 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.001344007, 2023-12-14T18:36:40.055 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000000000, 2023-12-14T18:36:40.055 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.000331749, 2023-12-14T18:36:40.055 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:40.055 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:40.055 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:40.056 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:40.056 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:40.056 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.056 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:40.056 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:40.056 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:40.056 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:40.056 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.057 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:36:40.057 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:36:40.057 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.057 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:40.057 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 214, 2023-12-14T18:36:40.057 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 202, 2023-12-14T18:36:40.058 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 1421440, 2023-12-14T18:36:40.058 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 114272, 2023-12-14T18:36:40.058 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-12-14T18:36:40.058 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-12-14T18:36:40.058 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.037202911, 2023-12-14T18:36:40.058 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.011750091, 2023-12-14T18:36:40.058 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.029265614, 2023-12-14T18:36:40.059 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.004085199, 2023-12-14T18:36:40.059 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:40.059 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 202, 2023-12-14T18:36:40.059 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.014775249, 2023-12-14T18:36:40.059 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000073144 2023-12-14T18:36:40.059 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.059 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:40.060 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:40.060 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:40.060 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:40.060 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.060 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 1421440, 2023-12-14T18:36:40.060 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 114272 2023-12-14T18:36:40.060 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.060 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:40.061 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 2, 2023-12-14T18:36:40.061 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 153, 2023-12-14T18:36:40.061 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 320, 2023-12-14T18:36:40.061 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 178656, 2023-12-14T18:36:40.061 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 5, 2023-12-14T18:36:40.061 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 3, 2023-12-14T18:36:40.061 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.010965071, 2023-12-14T18:36:40.062 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.007329059, 2023-12-14T18:36:40.062 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.001633269, 2023-12-14T18:36:40.062 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:40.062 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:40.062 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 150, 2023-12-14T18:36:40.062 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.013347079, 2023-12-14T18:36:40.062 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000088980 2023-12-14T18:36:40.063 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.063 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:40.063 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:40.063 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:40.063 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:40.063 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.063 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:36:40.064 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 178656 2023-12-14T18:36:40.064 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.064 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-12-14T18:36:40.064 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-12-14T18:36:40.064 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-12-14T18:36:40.064 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.064 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-12-14T18:36:40.065 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:40.065 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:40.065 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:40.065 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:40.065 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:40.065 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:40.065 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:40.066 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:40.066 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:40.066 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:40.066 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:40.066 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:40.066 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:40.066 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:40.066 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:40.067 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:40.067 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:40.067 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:40.067 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:40.067 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:40.067 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:40.067 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:40.067 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:40.068 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:40.068 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:40.068 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:40.068 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:40.068 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:40.068 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:40.068 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:40.069 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:40.069 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-12-14T18:36:40.069 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:40.069 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:40.069 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:40.069 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:40.069 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 148977, 2023-12-14T18:36:40.070 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 30, 2023-12-14T18:36:40.070 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:40.070 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:40.070 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-12-14T18:36:40.070 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-12-14T18:36:40.070 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:40.070 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:40.070 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:40.071 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:40.071 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-12-14T18:36:40.071 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-12-14T18:36:40.071 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:40.071 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:40.071 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:40.071 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-12-14T18:36:40.072 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-12-14T18:36:40.072 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-12-14T18:36:40.072 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:40.072 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-12-14T18:36:40.072 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:40.072 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-12-14T18:36:40.072 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:40.072 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:40.073 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-12-14T18:36:40.073 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-12-14T18:36:40.073 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-12-14T18:36:40.073 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 212, 2023-12-14T18:36:40.073 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 1388817, 2023-12-14T18:36:40.073 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:40.073 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 212, 2023-12-14T18:36:40.074 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-12-14T18:36:40.074 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-12-14T18:36:40.074 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 212, 2023-12-14T18:36:40.074 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1388817, 2023-12-14T18:36:40.074 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-12-14T18:36:40.074 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:40.074 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:40.074 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:40.075 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:40.075 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:40.075 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:36:40.418 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2023-12-14T18:36:40.418 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2023-12-14T18:36:40.418 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2023-12-14T18:36:40.418 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2023-12-14T18:36:40.418 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2023-12-14T18:36:40.418 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2023-12-14T18:36:40.419 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2023-12-14T18:36:40.419 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2023-12-14T18:36:40.419 DEBUG:tasks.check_counter:Getting stats from g 2023-12-14T18:36:40.419 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.g perf dump 2023-12-14T18:36:40.734 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:36:41.146 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:41 smithi093 ceph-mon[42070]: pgmap v317: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:41.146 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:41 smithi093 ceph-mon[42070]: osdmap e96: 12 total, 12 up, 12 in 2023-12-14T18:36:41.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:41 smithi097 ceph-mon[43486]: pgmap v317: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:41.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:41 smithi097 ceph-mon[43486]: osdmap e96: 12 total, 12 up, 12 in 2023-12-14T18:36:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:41 smithi160 ceph-mon[43840]: pgmap v317: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:41.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:41 smithi160 ceph-mon[43840]: osdmap e96: 12 total, 12 up, 12 in 2023-12-14T18:36:41.711 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:36:41.711 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:41.712 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 208, 2023-12-14T18:36:41.712 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 198, 2023-12-14T18:36:41.712 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 1417904, 2023-12-14T18:36:41.712 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 112640, 2023-12-14T18:36:41.712 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-12-14T18:36:41.712 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-12-14T18:36:41.712 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.037109740, 2023-12-14T18:36:41.713 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.011587829, 2023-12-14T18:36:41.713 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.027802895, 2023-12-14T18:36:41.713 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.004106087, 2023-12-14T18:36:41.713 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:41.713 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 197, 2023-12-14T18:36:41.713 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.013943673, 2023-12-14T18:36:41.714 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000070780 2023-12-14T18:36:41.714 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.714 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:41.714 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:41.714 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:41.714 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:41.714 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.715 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 1417904, 2023-12-14T18:36:41.715 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 112640 2023-12-14T18:36:41.715 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.715 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:41.715 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 6, 2023-12-14T18:36:41.715 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 153, 2023-12-14T18:36:41.715 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 3312, 2023-12-14T18:36:41.716 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 178752, 2023-12-14T18:36:41.716 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 5, 2023-12-14T18:36:41.716 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 3, 2023-12-14T18:36:41.716 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.012141586, 2023-12-14T18:36:41.716 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.007644096, 2023-12-14T18:36:41.716 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.002448303, 2023-12-14T18:36:41.716 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:41.717 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:41.717 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 151, 2023-12-14T18:36:41.717 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.025647310, 2023-12-14T18:36:41.717 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000169849 2023-12-14T18:36:41.717 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.717 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:41.718 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:41.718 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:41.718 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:41.718 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.718 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-12-14T18:36:41.718 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 178752 2023-12-14T18:36:41.718 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.719 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:41.719 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 0, 2023-12-14T18:36:41.719 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2, 2023-12-14T18:36:41.719 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 0, 2023-12-14T18:36:41.719 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 0, 2023-12-14T18:36:41.719 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-12-14T18:36:41.719 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-12-14T18:36:41.720 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.000361687, 2023-12-14T18:36:41.720 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000000000, 2023-12-14T18:36:41.720 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.000278209, 2023-12-14T18:36:41.720 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:41.720 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:41.720 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:41.720 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:41.721 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:41.721 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.721 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:41.721 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:41.721 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:41.721 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:41.721 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.722 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:36:41.722 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:36:41.722 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.722 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-12-14T18:36:41.722 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-12-14T18:36:41.722 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-12-14T18:36:41.722 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.723 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-12-14T18:36:41.723 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:41.723 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:41.723 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:41.723 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:41.723 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:41.723 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:41.724 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:41.724 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:41.724 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:41.724 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:41.724 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:41.724 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:41.724 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:41.725 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:41.725 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:41.725 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:41.725 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:41.725 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:41.725 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:41.725 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:41.726 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:41.726 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:41.726 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:41.726 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:41.726 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:41.726 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:41.726 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:41.726 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:41.727 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:41.727 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:41.727 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:41.727 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-12-14T18:36:41.727 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:41.727 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:41.727 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:41.728 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:41.728 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 148977, 2023-12-14T18:36:41.728 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 30, 2023-12-14T18:36:41.728 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:41.728 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:41.728 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-12-14T18:36:41.728 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-12-14T18:36:41.729 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:41.729 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:41.729 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:41.729 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:41.729 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-12-14T18:36:41.729 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-12-14T18:36:41.729 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:41.729 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:41.730 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:41.730 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-12-14T18:36:41.730 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-12-14T18:36:41.730 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-12-14T18:36:41.730 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:41.730 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-12-14T18:36:41.730 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:41.731 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-12-14T18:36:41.731 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:41.731 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:41.731 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-12-14T18:36:41.731 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-12-14T18:36:41.731 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-12-14T18:36:41.731 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 210, 2023-12-14T18:36:41.732 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 1388557, 2023-12-14T18:36:41.732 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:41.732 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 210, 2023-12-14T18:36:41.732 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-12-14T18:36:41.732 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-12-14T18:36:41.732 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 210, 2023-12-14T18:36:41.732 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1388557, 2023-12-14T18:36:41.733 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-12-14T18:36:41.733 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:41.733 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:41.733 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:41.733 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:41.733 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:41.733 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:36:42.083 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.g 2023-12-14T18:36:42.083 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.g 2023-12-14T18:36:42.084 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.g 2023-12-14T18:36:42.084 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.g 2023-12-14T18:36:42.084 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2023-12-14T18:36:42.084 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2023-12-14T18:36:42.084 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2023-12-14T18:36:42.084 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.g 2023-12-14T18:36:42.084 DEBUG:tasks.check_counter:Getting stats from j 2023-12-14T18:36:42.085 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.j perf dump 2023-12-14T18:36:42.399 INFO:teuthology.orchestra.run.smithi093.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.a/config 2023-12-14T18:36:43.121 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:43 smithi093 ceph-mon[42070]: pgmap v319: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:43.328 INFO:teuthology.orchestra.run.smithi093.stdout:{ 2023-12-14T18:36:43.329 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:43.329 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 0, 2023-12-14T18:36:43.329 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 2, 2023-12-14T18:36:43.329 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 0, 2023-12-14T18:36:43.329 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 0, 2023-12-14T18:36:43.329 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 4, 2023-12-14T18:36:43.330 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 0, 2023-12-14T18:36:43.330 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.001115372, 2023-12-14T18:36:43.330 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.000000000, 2023-12-14T18:36:43.330 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.000222011, 2023-12-14T18:36:43.330 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:43.330 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:43.331 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:43.331 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:43.331 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:43.331 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.331 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:43.331 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:43.331 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:43.332 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:43.332 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.332 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:36:43.332 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:36:43.332 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.332 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:43.332 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 212, 2023-12-14T18:36:43.333 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 202, 2023-12-14T18:36:43.333 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 1418992, 2023-12-14T18:36:43.333 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 115008, 2023-12-14T18:36:43.333 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 2, 2023-12-14T18:36:43.333 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 1, 2023-12-14T18:36:43.333 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.036680920, 2023-12-14T18:36:43.334 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.011769926, 2023-12-14T18:36:43.334 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.028431654, 2023-12-14T18:36:43.334 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.004166011, 2023-12-14T18:36:43.334 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:43.334 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 201, 2023-12-14T18:36:43.334 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.013852165, 2023-12-14T18:36:43.335 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000068916 2023-12-14T18:36:43.335 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.335 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:43.335 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:43.335 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:43.335 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:43.335 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.336 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 1418992, 2023-12-14T18:36:43.336 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 115008 2023-12-14T18:36:43.336 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.336 INFO:teuthology.orchestra.run.smithi093.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:43.336 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_messages": 6, 2023-12-14T18:36:43.336 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages": 156, 2023-12-14T18:36:43.336 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_bytes": 3312, 2023-12-14T18:36:43.337 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_bytes": 179280, 2023-12-14T18:36:43.337 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_created_connections": 5, 2023-12-14T18:36:43.337 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_active_connections": 3, 2023-12-14T18:36:43.337 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_total_time": 0.011708840, 2023-12-14T18:36:43.337 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_send_time": 0.007505796, 2023-12-14T18:36:43.337 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_recv_time": 0.002174003, 2023-12-14T18:36:43.338 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:43.338 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:43.338 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 154, 2023-12-14T18:36:43.338 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.022941536, 2023-12-14T18:36:43.338 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000148971 2023-12-14T18:36:43.338 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.338 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:43.339 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:43.339 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:43.339 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:43.339 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.339 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-12-14T18:36:43.339 INFO:teuthology.orchestra.run.smithi093.stdout: "msgr_send_encrypted_bytes": 179280 2023-12-14T18:36:43.339 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.340 INFO:teuthology.orchestra.run.smithi093.stdout: "cct": { 2023-12-14T18:36:43.340 INFO:teuthology.orchestra.run.smithi093.stdout: "total_workers": 0, 2023-12-14T18:36:43.340 INFO:teuthology.orchestra.run.smithi093.stdout: "unhealthy_workers": 0 2023-12-14T18:36:43.340 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.340 INFO:teuthology.orchestra.run.smithi093.stdout: "mempool": { 2023-12-14T18:36:43.340 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:43.340 INFO:teuthology.orchestra.run.smithi093.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:43.341 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:43.341 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:43.341 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:43.341 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:43.341 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:43.341 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:43.341 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:43.342 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:43.342 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:43.342 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:43.342 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:43.342 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:43.342 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:43.342 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:43.343 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:43.343 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:43.343 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:43.343 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:43.343 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:43.343 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:43.343 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:43.344 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:43.344 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:43.344 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:43.344 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:43.344 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:43.344 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:43.344 INFO:teuthology.orchestra.run.smithi093.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:43.345 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:43.345 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_items": 0, 2023-12-14T18:36:43.345 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:43.345 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:43.345 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:43.345 INFO:teuthology.orchestra.run.smithi093.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:43.345 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_bytes": 148994, 2023-12-14T18:36:43.346 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_anon_items": 31, 2023-12-14T18:36:43.346 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:43.346 INFO:teuthology.orchestra.run.smithi093.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:43.346 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_bytes": 0, 2023-12-14T18:36:43.346 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_items": 0, 2023-12-14T18:36:43.346 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:43.346 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:43.347 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:43.347 INFO:teuthology.orchestra.run.smithi093.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:43.347 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_bytes": 0, 2023-12-14T18:36:43.347 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_items": 0, 2023-12-14T18:36:43.347 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:43.347 INFO:teuthology.orchestra.run.smithi093.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:43.347 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:43.348 INFO:teuthology.orchestra.run.smithi093.stdout: "pgmap_items": 0, 2023-12-14T18:36:43.348 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_bytes": 928, 2023-12-14T18:36:43.348 INFO:teuthology.orchestra.run.smithi093.stdout: "mds_co_items": 2, 2023-12-14T18:36:43.348 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:43.348 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_1_items": 0, 2023-12-14T18:36:43.348 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:43.348 INFO:teuthology.orchestra.run.smithi093.stdout: "unittest_2_items": 0 2023-12-14T18:36:43.349 INFO:teuthology.orchestra.run.smithi093.stdout: }, 2023-12-14T18:36:43.349 INFO:teuthology.orchestra.run.smithi093.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:43.349 INFO:teuthology.orchestra.run.smithi093.stdout: "val": 0, 2023-12-14T18:36:43.349 INFO:teuthology.orchestra.run.smithi093.stdout: "max": 104857600, 2023-12-14T18:36:43.349 INFO:teuthology.orchestra.run.smithi093.stdout: "get_started": 0, 2023-12-14T18:36:43.349 INFO:teuthology.orchestra.run.smithi093.stdout: "get": 214, 2023-12-14T18:36:43.349 INFO:teuthology.orchestra.run.smithi093.stdout: "get_sum": 1389077, 2023-12-14T18:36:43.350 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:43.350 INFO:teuthology.orchestra.run.smithi093.stdout: "get_or_fail_success": 214, 2023-12-14T18:36:43.350 INFO:teuthology.orchestra.run.smithi093.stdout: "take": 0, 2023-12-14T18:36:43.350 INFO:teuthology.orchestra.run.smithi093.stdout: "take_sum": 0, 2023-12-14T18:36:43.350 INFO:teuthology.orchestra.run.smithi093.stdout: "put": 214, 2023-12-14T18:36:43.350 INFO:teuthology.orchestra.run.smithi093.stdout: "put_sum": 1389077, 2023-12-14T18:36:43.350 INFO:teuthology.orchestra.run.smithi093.stdout: "wait": { 2023-12-14T18:36:43.350 INFO:teuthology.orchestra.run.smithi093.stdout: "avgcount": 0, 2023-12-14T18:36:43.351 INFO:teuthology.orchestra.run.smithi093.stdout: "sum": 0.000000000, 2023-12-14T18:36:43.351 INFO:teuthology.orchestra.run.smithi093.stdout: "avgtime": 0.000000000 2023-12-14T18:36:43.351 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:43.351 INFO:teuthology.orchestra.run.smithi093.stdout: } 2023-12-14T18:36:43.351 INFO:teuthology.orchestra.run.smithi093.stdout:} 2023-12-14T18:36:43.352 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:43 smithi097 ceph-mon[43486]: pgmap v319: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:43.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:43 smithi160 ceph-mon[43840]: pgmap v319: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:43.699 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2023-12-14T18:36:43.699 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2023-12-14T18:36:43.699 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2023-12-14T18:36:43.699 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2023-12-14T18:36:43.700 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2023-12-14T18:36:43.700 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2023-12-14T18:36:43.700 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2023-12-14T18:36:43.700 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2023-12-14T18:36:43.700 DEBUG:tasks.check_counter:Getting stats from b 2023-12-14T18:36:43.700 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.b perf dump 2023-12-14T18:36:44.145 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:36:45.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:45 smithi093 ceph-mon[42070]: pgmap v320: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:45.319 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-12-14T18:36:45.319 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:45.319 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 32510, 2023-12-14T18:36:45.320 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 32736, 2023-12-14T18:36:45.320 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 75736764, 2023-12-14T18:36:45.320 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 166847711, 2023-12-14T18:36:45.320 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 58, 2023-12-14T18:36:45.320 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2023-12-14T18:36:45.320 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 2.103021941, 2023-12-14T18:36:45.321 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.768984870, 2023-12-14T18:36:45.321 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.624256103, 2023-12-14T18:36:45.321 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.739990333, 2023-12-14T18:36:45.321 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:45.321 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 32735, 2023-12-14T18:36:45.321 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 44.408237120, 2023-12-14T18:36:45.322 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001356598 2023-12-14T18:36:45.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.322 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:45.322 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 15695, 2023-12-14T18:36:45.322 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002363173, 2023-12-14T18:36:45.322 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000150 2023-12-14T18:36:45.322 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.323 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1562128, 2023-12-14T18:36:45.323 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 126192 2023-12-14T18:36:45.323 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.323 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:45.323 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 126420, 2023-12-14T18:36:45.323 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 211167, 2023-12-14T18:36:45.324 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 45831800, 2023-12-14T18:36:45.324 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 285472790, 2023-12-14T18:36:45.324 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 15, 2023-12-14T18:36:45.324 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2023-12-14T18:36:45.324 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 6.907777709, 2023-12-14T18:36:45.324 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 2.876434417, 2023-12-14T18:36:45.324 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 242.344038777, 2023-12-14T18:36:45.325 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.954200120, 2023-12-14T18:36:45.325 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:45.325 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 211165, 2023-12-14T18:36:45.325 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 82.589061323, 2023-12-14T18:36:45.325 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000391111 2023-12-14T18:36:45.325 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.326 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:45.326 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 248020, 2023-12-14T18:36:45.326 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.038002728, 2023-12-14T18:36:45.326 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000153 2023-12-14T18:36:45.326 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.326 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 91328, 2023-12-14T18:36:45.326 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 456976 2023-12-14T18:36:45.327 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.327 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:45.327 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 45292, 2023-12-14T18:36:45.327 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 43172, 2023-12-14T18:36:45.327 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 86129489, 2023-12-14T18:36:45.327 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 130687072, 2023-12-14T18:36:45.327 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 18, 2023-12-14T18:36:45.328 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 7, 2023-12-14T18:36:45.328 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 2.526219657, 2023-12-14T18:36:45.328 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.833158227, 2023-12-14T18:36:45.328 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.793753022, 2023-12-14T18:36:45.328 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.938335614, 2023-12-14T18:36:45.328 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:45.329 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 43170, 2023-12-14T18:36:45.329 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 61.762415320, 2023-12-14T18:36:45.329 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001430679 2023-12-14T18:36:45.329 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.329 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:45.329 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 21950, 2023-12-14T18:36:45.330 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.003553396, 2023-12-14T18:36:45.330 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000161 2023-12-14T18:36:45.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.330 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 304624, 2023-12-14T18:36:45.330 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 85552 2023-12-14T18:36:45.330 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.330 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2023-12-14T18:36:45.331 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2023-12-14T18:36:45.331 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2023-12-14T18:36:45.331 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.331 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2023-12-14T18:36:45.331 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2023-12-14T18:36:45.331 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2023-12-14T18:36:45.331 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4420, 2023-12-14T18:36:45.332 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 89.873905777, 2023-12-14T18:36:45.332 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.020333462 2023-12-14T18:36:45.332 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:45.332 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.332 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2023-12-14T18:36:45.332 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2023-12-14T18:36:45.332 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2023-12-14T18:36:45.333 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 20118, 2023-12-14T18:36:45.333 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 10.051595930, 2023-12-14T18:36:45.333 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000499631 2023-12-14T18:36:45.333 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:45.333 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.333 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2023-12-14T18:36:45.333 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 58275, 2023-12-14T18:36:45.334 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 48183, 2023-12-14T18:36:45.334 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2023-12-14T18:36:45.334 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 48183, 2023-12-14T18:36:45.334 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 211.373640657, 2023-12-14T18:36:45.334 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.004386892 2023-12-14T18:36:45.334 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.334 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2023-12-14T18:36:45.335 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 10092, 2023-12-14T18:36:45.335 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 724, 2023-12-14T18:36:45.335 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2023-12-14T18:36:45.335 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 10348, 2023-12-14T18:36:45.335 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 144, 2023-12-14T18:36:45.335 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 74, 2023-12-14T18:36:45.335 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 13771, 2023-12-14T18:36:45.336 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 9619, 2023-12-14T18:36:45.336 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 4142, 2023-12-14T18:36:45.336 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 10, 2023-12-14T18:36:45.336 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 28, 2023-12-14T18:36:45.336 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 40775, 2023-12-14T18:36:45.336 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 7, 2023-12-14T18:36:45.336 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 7, 2023-12-14T18:36:45.337 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 5, 2023-12-14T18:36:45.337 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 72460, 2023-12-14T18:36:45.337 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 61898, 2023-12-14T18:36:45.337 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 10064, 2023-12-14T18:36:45.337 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 3, 2023-12-14T18:36:45.337 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:36:45.337 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2023-12-14T18:36:45.338 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2023-12-14T18:36:45.338 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2023-12-14T18:36:45.338 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2023-12-14T18:36:45.338 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 598, 2023-12-14T18:36:45.338 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 35301, 2023-12-14T18:36:45.338 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 408, 2023-12-14T18:36:45.338 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2023-12-14T18:36:45.339 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2023-12-14T18:36:45.339 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2023-12-14T18:36:45.339 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2023-12-14T18:36:45.339 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 7, 2023-12-14T18:36:45.339 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 246, 2023-12-14T18:36:45.339 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 6, 2023-12-14T18:36:45.339 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2023-12-14T18:36:45.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2023-12-14T18:36:45.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:36:45.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:36:45.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2023-12-14T18:36:45.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:36:45.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2023-12-14T18:36:45.340 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2023-12-14T18:36:45.341 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 2, 2023-12-14T18:36:45.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 3428, 2023-12-14T18:36:45.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 36506, 2023-12-14T18:36:45.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 1, 2023-12-14T18:36:45.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 2343, 2023-12-14T18:36:45.341 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:36:45.341 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 36542, 2023-12-14T18:36:45.342 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 33115, 2023-12-14T18:36:45.342 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 863, 2023-12-14T18:36:45.342 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 27949 2023-12-14T18:36:45.342 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.342 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2023-12-14T18:36:45.342 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 1014, 2023-12-14T18:36:45.342 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 3628, 2023-12-14T18:36:45.343 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 1814, 2023-12-14T18:36:45.343 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 1830, 2023-12-14T18:36:45.343 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 1066, 2023-12-14T18:36:45.343 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 9, 2023-12-14T18:36:45.343 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2023-12-14T18:36:45.343 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:36:45.343 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 13198, 2023-12-14T18:36:45.344 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 13189, 2023-12-14T18:36:45.344 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2023-12-14T18:36:45.344 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2023-12-14T18:36:45.344 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2023-12-14T18:36:45.344 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:36:45.344 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:36:45.344 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2023-12-14T18:36:45.344 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2023-12-14T18:36:45.345 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:36:45.345 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 612, 2023-12-14T18:36:45.345 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2023-12-14T18:36:45.345 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 222, 2023-12-14T18:36:45.345 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2023-12-14T18:36:45.345 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2023-12-14T18:36:45.345 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.346 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2023-12-14T18:36:45.346 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 88798, 2023-12-14T18:36:45.346 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 88632, 2023-12-14T18:36:45.346 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 88632, 2023-12-14T18:36:45.346 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 166, 2023-12-14T18:36:45.346 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2023-12-14T18:36:45.346 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 38, 2023-12-14T18:36:45.347 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 148, 2023-12-14T18:36:45.347 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 147, 2023-12-14T18:36:45.347 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 147, 2023-12-14T18:36:45.347 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 3, 2023-12-14T18:36:45.347 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2023-12-14T18:36:45.347 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 2, 2023-12-14T18:36:45.347 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 348208657, 2023-12-14T18:36:45.348 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 348780231, 2023-12-14T18:36:45.348 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2023-12-14T18:36:45.348 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2023-12-14T18:36:45.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2250, 2023-12-14T18:36:45.348 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 198.484184132, 2023-12-14T18:36:45.348 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.088215192 2023-12-14T18:36:45.348 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.349 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2023-12-14T18:36:45.349 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.349 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2023-12-14T18:36:45.349 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 13744, 2023-12-14T18:36:45.349 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 61841, 2023-12-14T18:36:45.349 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 48097, 2023-12-14T18:36:45.349 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 462, 2023-12-14T18:36:45.350 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 3270, 2023-12-14T18:36:45.350 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 2808, 2023-12-14T18:36:45.350 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 13771, 2023-12-14T18:36:45.350 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 77370, 2023-12-14T18:36:45.350 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 63599, 2023-12-14T18:36:45.350 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 7, 2023-12-14T18:36:45.350 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 39358, 2023-12-14T18:36:45.351 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 39351, 2023-12-14T18:36:45.351 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 275064, 2023-12-14T18:36:45.351 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 207156 2023-12-14T18:36:45.351 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.351 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2023-12-14T18:36:45.351 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 59510, 2023-12-14T18:36:45.351 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 17, 2023-12-14T18:36:45.351 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 58275, 2023-12-14T18:36:45.352 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 79, 2023-12-14T18:36:45.352 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 99, 2023-12-14T18:36:45.352 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2023-12-14T18:36:45.352 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 20928, 2023-12-14T18:36:45.352 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 70.856375049, 2023-12-14T18:36:45.352 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003385721 2023-12-14T18:36:45.352 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.353 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2023-12-14T18:36:45.353 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 244, 2023-12-14T18:36:45.353 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.409459782, 2023-12-14T18:36:45.353 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001678113 2023-12-14T18:36:45.353 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.353 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2023-12-14T18:36:45.353 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.354 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.354 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.354 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2023-12-14T18:36:45.354 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.354 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.354 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.354 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.355 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2023-12-14T18:36:45.355 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 11967, 2023-12-14T18:36:45.355 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 36.430980160, 2023-12-14T18:36:45.355 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003044286 2023-12-14T18:36:45.355 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.355 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2023-12-14T18:36:45.355 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.356 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.356 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.356 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2023-12-14T18:36:45.356 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.356 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.356 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.356 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.357 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2023-12-14T18:36:45.357 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.357 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.357 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.357 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.357 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2023-12-14T18:36:45.357 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.358 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.358 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.358 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.358 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2023-12-14T18:36:45.358 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.358 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.358 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.359 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2023-12-14T18:36:45.359 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10, 2023-12-14T18:36:45.359 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.004134390, 2023-12-14T18:36:45.359 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000413439 2023-12-14T18:36:45.359 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.359 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2023-12-14T18:36:45.359 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 129, 2023-12-14T18:36:45.360 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.124391882, 2023-12-14T18:36:45.360 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000964278 2023-12-14T18:36:45.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.360 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2023-12-14T18:36:45.360 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.360 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.360 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.360 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.361 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2023-12-14T18:36:45.361 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 10, 2023-12-14T18:36:45.361 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 3.243677171, 2023-12-14T18:36:45.361 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.324367717 2023-12-14T18:36:45.361 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.361 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2023-12-14T18:36:45.361 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.362 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.362 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.362 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2023-12-14T18:36:45.362 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1608, 2023-12-14T18:36:45.362 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 5.611476753, 2023-12-14T18:36:45.362 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003489724 2023-12-14T18:36:45.362 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.363 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2023-12-14T18:36:45.363 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2023-12-14T18:36:45.363 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.007107372, 2023-12-14T18:36:45.363 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001776843 2023-12-14T18:36:45.363 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.363 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2023-12-14T18:36:45.363 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.363 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.364 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.364 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2023-12-14T18:36:45.364 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.364 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.364 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.364 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.365 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2023-12-14T18:36:45.365 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 116, 2023-12-14T18:36:45.365 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.409436223, 2023-12-14T18:36:45.365 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.003529622 2023-12-14T18:36:45.365 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.365 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2023-12-14T18:36:45.365 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2023-12-14T18:36:45.365 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.020651064, 2023-12-14T18:36:45.366 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005162766 2023-12-14T18:36:45.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.366 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2023-12-14T18:36:45.366 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 21, 2023-12-14T18:36:45.366 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.027466710, 2023-12-14T18:36:45.366 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001307938 2023-12-14T18:36:45.366 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.366 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2023-12-14T18:36:45.367 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 5, 2023-12-14T18:36:45.367 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.013378805, 2023-12-14T18:36:45.367 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002675761 2023-12-14T18:36:45.367 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.367 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2023-12-14T18:36:45.367 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.367 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.368 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.368 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2023-12-14T18:36:45.368 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 22, 2023-12-14T18:36:45.368 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 10.061310213, 2023-12-14T18:36:45.368 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.457332282 2023-12-14T18:36:45.368 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.368 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2023-12-14T18:36:45.369 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.369 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.369 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.369 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.369 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2023-12-14T18:36:45.369 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 34, 2023-12-14T18:36:45.369 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.085698649, 2023-12-14T18:36:45.370 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002520548 2023-12-14T18:36:45.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.370 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2023-12-14T18:36:45.370 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.370 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.370 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.370 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.370 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2023-12-14T18:36:45.371 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 13081, 2023-12-14T18:36:45.371 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 84.068096434, 2023-12-14T18:36:45.371 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.006426733 2023-12-14T18:36:45.371 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.371 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:36:45.371 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:36:45.371 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2023-12-14T18:36:45.371 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.372 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.372 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.372 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:45.372 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.372 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2023-12-14T18:36:45.372 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2023-12-14T18:36:45.372 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 6, 2023-12-14T18:36:45.372 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 4, 2023-12-14T18:36:45.373 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 2, 2023-12-14T18:36:45.373 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2023-12-14T18:36:45.373 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 1250, 2023-12-14T18:36:45.373 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 625, 2023-12-14T18:36:45.373 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 601 2023-12-14T18:36:45.373 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.373 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2023-12-14T18:36:45.373 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 480, 2023-12-14T18:36:45.374 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 480, 2023-12-14T18:36:45.374 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:45.374 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:45.374 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:45.374 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:45.374 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:45.374 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:45.374 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:45.375 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:45.375 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:45.375 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:45.375 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:45.375 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:45.375 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:45.375 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:45.375 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:45.376 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:45.376 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:45.376 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:45.376 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:45.376 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:45.376 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:45.376 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:45.376 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:45.376 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:45.377 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:45.377 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:45.377 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:45.377 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:45.377 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:45.377 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2023-12-14T18:36:45.377 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:45.378 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:45.378 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:45.378 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:45.378 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 737694, 2023-12-14T18:36:45.378 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 11171, 2023-12-14T18:36:45.378 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 616, 2023-12-14T18:36:45.378 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 7, 2023-12-14T18:36:45.378 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2023-12-14T18:36:45.379 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2023-12-14T18:36:45.379 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:45.379 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:45.379 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:45.379 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:45.379 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 9200, 2023-12-14T18:36:45.379 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 175, 2023-12-14T18:36:45.379 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:45.380 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:45.380 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:45.380 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2023-12-14T18:36:45.380 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 40808609, 2023-12-14T18:36:45.380 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 329322, 2023-12-14T18:36:45.380 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:45.380 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2023-12-14T18:36:45.380 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:45.381 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2023-12-14T18:36:45.381 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.381 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2023-12-14T18:36:45.381 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2023-12-14T18:36:45.381 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2023-12-14T18:36:45.381 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 86537, 2023-12-14T18:36:45.381 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 420844427, 2023-12-14T18:36:45.381 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 37, 2023-12-14T18:36:45.382 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 86537, 2023-12-14T18:36:45.382 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2023-12-14T18:36:45.382 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 86537, 2023-12-14T18:36:45.382 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 29550.223746373, 2023-12-14T18:36:45.382 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.341475019 2023-12-14T18:36:45.382 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.382 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2023-12-14T18:36:45.382 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2023-12-14T18:36:45.383 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 86537, 2023-12-14T18:36:45.383 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 171240 2023-12-14T18:36:45.383 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.383 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 86537, 2023-12-14T18:36:45.383 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 964, 2023-12-14T18:36:45.383 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 85573, 2023-12-14T18:36:45.383 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2023-12-14T18:36:45.383 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2023-12-14T18:36:45.383 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 9087, 2023-12-14T18:36:45.384 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 45078, 2023-12-14T18:36:45.384 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 240, 2023-12-14T18:36:45.384 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 2490, 2023-12-14T18:36:45.384 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 220, 2023-12-14T18:36:45.384 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2023-12-14T18:36:45.384 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2023-12-14T18:36:45.384 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2023-12-14T18:36:45.384 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 13, 2023-12-14T18:36:45.385 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 27286, 2023-12-14T18:36:45.385 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2023-12-14T18:36:45.385 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2023-12-14T18:36:45.385 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2023-12-14T18:36:45.385 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 2, 2023-12-14T18:36:45.385 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 69141, 2023-12-14T18:36:45.385 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:36:45.385 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2023-12-14T18:36:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:36:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2023-12-14T18:36:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2023-12-14T18:36:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2023-12-14T18:36:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:36:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2023-12-14T18:36:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:36:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 1, 2023-12-14T18:36:45.386 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2023-12-14T18:36:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2023-12-14T18:36:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2023-12-14T18:36:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2023-12-14T18:36:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2023-12-14T18:36:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2023-12-14T18:36:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2023-12-14T18:36:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2023-12-14T18:36:45.387 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2023-12-14T18:36:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2023-12-14T18:36:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2023-12-14T18:36:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2023-12-14T18:36:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2023-12-14T18:36:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2023-12-14T18:36:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2023-12-14T18:36:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2023-12-14T18:36:45.388 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 96, 2023-12-14T18:36:45.389 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2023-12-14T18:36:45.389 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 17, 2023-12-14T18:36:45.389 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 12, 2023-12-14T18:36:45.389 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 12, 2023-12-14T18:36:45.389 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2023-12-14T18:36:45.389 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2023-12-14T18:36:45.389 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 15546, 2023-12-14T18:36:45.390 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 1448, 2023-12-14T18:36:45.390 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 688 2023-12-14T18:36:45.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.390 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2023-12-14T18:36:45.390 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2023-12-14T18:36:45.390 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:36:45.390 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2023-12-14T18:36:45.390 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2023-12-14T18:36:45.390 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.391 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2023-12-14T18:36:45.391 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 59126 2023-12-14T18:36:45.391 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.391 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2023-12-14T18:36:45.391 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2023-12-14T18:36:45.391 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 16, 2023-12-14T18:36:45.391 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2023-12-14T18:36:45.391 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 12, 2023-12-14T18:36:45.391 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 13189, 2023-12-14T18:36:45.392 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2023-12-14T18:36:45.392 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.392 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:45.392 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:45.392 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-12-14T18:36:45.392 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:45.392 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 204218, 2023-12-14T18:36:45.392 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 189255490, 2023-12-14T18:36:45.393 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:45.393 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 204218, 2023-12-14T18:36:45.393 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:45.393 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:45.393 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 204218, 2023-12-14T18:36:45.393 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 189255490, 2023-12-14T18:36:45.393 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:45.393 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.394 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.394 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.394 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:45.394 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.394 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2023-12-14T18:36:45.394 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:45.394 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-12-14T18:36:45.394 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:45.394 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2023-12-14T18:36:45.395 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2023-12-14T18:36:45.395 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:45.395 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:45.395 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 86537, 2023-12-14T18:36:45.395 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 422443820, 2023-12-14T18:36:45.395 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 86537, 2023-12-14T18:36:45.395 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 422443820, 2023-12-14T18:36:45.395 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2023-12-14T18:36:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2023-12-14T18:36:45.396 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:45.397 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2023-12-14T18:36:45.397 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2023-12-14T18:36:45.397 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:45.397 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:45.397 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 86537, 2023-12-14T18:36:45.397 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 86537, 2023-12-14T18:36:45.397 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 86537, 2023-12-14T18:36:45.397 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 86537, 2023-12-14T18:36:45.398 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:45.398 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.398 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.398 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.398 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:45.398 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.398 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:36:45.398 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:45.399 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2023-12-14T18:36:45.399 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:45.399 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 13189, 2023-12-14T18:36:45.399 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1610977, 2023-12-14T18:36:45.399 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:45.399 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 13189, 2023-12-14T18:36:45.399 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:45.399 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:45.399 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 240, 2023-12-14T18:36:45.400 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1610977, 2023-12-14T18:36:45.400 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:45.400 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.400 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.400 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.400 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:45.400 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:45.400 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x5623c25ee0a0": { 2023-12-14T18:36:45.400 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:45.401 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2023-12-14T18:36:45.401 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:45.401 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 88798, 2023-12-14T18:36:45.401 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 344585927, 2023-12-14T18:36:45.401 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:45.401 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 88798, 2023-12-14T18:36:45.401 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:45.401 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:45.402 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2250, 2023-12-14T18:36:45.402 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 344585927, 2023-12-14T18:36:45.402 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:45.402 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:45.402 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:45.402 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:45.402 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:45.402 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:45.402 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-12-14T18:36:45.403 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:45 smithi097 ceph-mon[43486]: pgmap v320: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:45.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:45 smithi160 ceph-mon[43840]: pgmap v320: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 0 B/s wr, 8 op/s 2023-12-14T18:36:45.731 INFO:tasks.check_counter:Daemon mds.b mds.exported=598 2023-12-14T18:36:45.732 INFO:tasks.check_counter:Daemon mds.b mds.imported=408 2023-12-14T18:36:45.732 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=1014 2023-12-14T18:36:45.732 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=3628 2023-12-14T18:36:45.732 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2023-12-14T18:36:45.732 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=10 2023-12-14T18:36:45.732 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=4 2023-12-14T18:36:45.732 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=144 2023-12-14T18:36:45.733 DEBUG:tasks.check_counter:Getting stats from e 2023-12-14T18:36:45.733 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.e perf dump 2023-12-14T18:36:46.046 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:36:47.043 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-12-14T18:36:47.044 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:47.044 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 20473, 2023-12-14T18:36:47.044 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 20261, 2023-12-14T18:36:47.044 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 21927488, 2023-12-14T18:36:47.044 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 168051741, 2023-12-14T18:36:47.044 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 9, 2023-12-14T18:36:47.045 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2023-12-14T18:36:47.045 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 1.527050726, 2023-12-14T18:36:47.045 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.636665408, 2023-12-14T18:36:47.045 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.529640366, 2023-12-14T18:36:47.045 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.396997122, 2023-12-14T18:36:47.045 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:47.046 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 20261, 2023-12-14T18:36:47.046 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 4.426359370, 2023-12-14T18:36:47.046 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000218466 2023-12-14T18:36:47.046 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.046 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:47.046 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 15789, 2023-12-14T18:36:47.047 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002672771, 2023-12-14T18:36:47.047 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000169 2023-12-14T18:36:47.047 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.047 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 160, 2023-12-14T18:36:47.047 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 299376 2023-12-14T18:36:47.048 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.048 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:47.048 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 98667, 2023-12-14T18:36:47.048 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 165514, 2023-12-14T18:36:47.048 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 33151482, 2023-12-14T18:36:47.048 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 263052632, 2023-12-14T18:36:47.049 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 12, 2023-12-14T18:36:47.049 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2023-12-14T18:36:47.049 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 5.712107864, 2023-12-14T18:36:47.049 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 2.531513905, 2023-12-14T18:36:47.049 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 290.824159125, 2023-12-14T18:36:47.049 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.529553522, 2023-12-14T18:36:47.050 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:47.050 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 165514, 2023-12-14T18:36:47.050 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 22.146403617, 2023-12-14T18:36:47.050 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000133803 2023-12-14T18:36:47.050 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.050 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:47.050 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 205894, 2023-12-14T18:36:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.032748617, 2023-12-14T18:36:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000159 2023-12-14T18:36:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1506480, 2023-12-14T18:36:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 123888 2023-12-14T18:36:47.051 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 26018, 2023-12-14T18:36:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 26917, 2023-12-14T18:36:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 77614154, 2023-12-14T18:36:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 204398115, 2023-12-14T18:36:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 13, 2023-12-14T18:36:47.052 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 8, 2023-12-14T18:36:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 1.794050729, 2023-12-14T18:36:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.670286748, 2023-12-14T18:36:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.616823609, 2023-12-14T18:36:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.510633569, 2023-12-14T18:36:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:47.053 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 26912, 2023-12-14T18:36:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 5.055844997, 2023-12-14T18:36:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000187865 2023-12-14T18:36:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 15776, 2023-12-14T18:36:47.054 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.002399639, 2023-12-14T18:36:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000152 2023-12-14T18:36:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 4080, 2023-12-14T18:36:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 120128 2023-12-14T18:36:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2023-12-14T18:36:47.055 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2023-12-14T18:36:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2023-12-14T18:36:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2023-12-14T18:36:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2023-12-14T18:36:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2023-12-14T18:36:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2029, 2023-12-14T18:36:47.056 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 117.436654315, 2023-12-14T18:36:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.057879080 2023-12-14T18:36:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2023-12-14T18:36:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2023-12-14T18:36:47.057 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2023-12-14T18:36:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 31385, 2023-12-14T18:36:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 27.924144189, 2023-12-14T18:36:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000889728 2023-12-14T18:36:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2023-12-14T18:36:47.058 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 57917, 2023-12-14T18:36:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 50262, 2023-12-14T18:36:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2023-12-14T18:36:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 50262, 2023-12-14T18:36:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 291.943291807, 2023-12-14T18:36:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005808429 2023-12-14T18:36:47.059 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2023-12-14T18:36:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 7655, 2023-12-14T18:36:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 362, 2023-12-14T18:36:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2023-12-14T18:36:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 871, 2023-12-14T18:36:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 106, 2023-12-14T18:36:47.060 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 12, 2023-12-14T18:36:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 49284, 2023-12-14T18:36:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 3045, 2023-12-14T18:36:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 34232, 2023-12-14T18:36:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 12007, 2023-12-14T18:36:47.061 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 44932, 2023-12-14T18:36:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 14576, 2023-12-14T18:36:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 1, 2023-12-14T18:36:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 1, 2023-12-14T18:36:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 4, 2023-12-14T18:36:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 79714, 2023-12-14T18:36:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 71787, 2023-12-14T18:36:47.065 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 7328, 2023-12-14T18:36:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 12, 2023-12-14T18:36:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:36:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2023-12-14T18:36:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2023-12-14T18:36:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2023-12-14T18:36:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2023-12-14T18:36:47.066 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 198, 2023-12-14T18:36:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 11209, 2023-12-14T18:36:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 417, 2023-12-14T18:36:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2023-12-14T18:36:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2023-12-14T18:36:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2023-12-14T18:36:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2023-12-14T18:36:47.067 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 3146, 2023-12-14T18:36:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 246, 2023-12-14T18:36:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 6, 2023-12-14T18:36:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2023-12-14T18:36:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2023-12-14T18:36:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:36:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:36:47.068 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2023-12-14T18:36:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:36:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2023-12-14T18:36:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2023-12-14T18:36:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 6, 2023-12-14T18:36:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 215, 2023-12-14T18:36:47.069 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 30958, 2023-12-14T18:36:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:36:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 1978, 2023-12-14T18:36:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:36:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 26374, 2023-12-14T18:36:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 26174, 2023-12-14T18:36:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 291, 2023-12-14T18:36:47.070 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 32502 2023-12-14T18:36:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2023-12-14T18:36:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 806, 2023-12-14T18:36:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 3065, 2023-12-14T18:36:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 1535, 2023-12-14T18:36:47.071 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 1604, 2023-12-14T18:36:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 807, 2023-12-14T18:36:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 2, 2023-12-14T18:36:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2023-12-14T18:36:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:36:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 20458, 2023-12-14T18:36:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 20456, 2023-12-14T18:36:47.072 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2023-12-14T18:36:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2023-12-14T18:36:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2023-12-14T18:36:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:36:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:36:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2023-12-14T18:36:47.073 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2023-12-14T18:36:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:36:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 199, 2023-12-14T18:36:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2023-12-14T18:36:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 124, 2023-12-14T18:36:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2023-12-14T18:36:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2023-12-14T18:36:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.074 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2023-12-14T18:36:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 76367, 2023-12-14T18:36:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2023-12-14T18:36:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2023-12-14T18:36:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 76367, 2023-12-14T18:36:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2023-12-14T18:36:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2023-12-14T18:36:47.075 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 108, 2023-12-14T18:36:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2023-12-14T18:36:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2023-12-14T18:36:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 108, 2023-12-14T18:36:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2023-12-14T18:36:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2023-12-14T18:36:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 4194304, 2023-12-14T18:36:47.076 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 383203012, 2023-12-14T18:36:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 0, 2023-12-14T18:36:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2023-12-14T18:36:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1210, 2023-12-14T18:36:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 191.901096955, 2023-12-14T18:36:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.158595947 2023-12-14T18:36:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.077 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 0 2023-12-14T18:36:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2023-12-14T18:36:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 29534, 2023-12-14T18:36:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 59124, 2023-12-14T18:36:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 29590, 2023-12-14T18:36:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 1041, 2023-12-14T18:36:47.078 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 2875, 2023-12-14T18:36:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 1834, 2023-12-14T18:36:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 49284, 2023-12-14T18:36:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 82473, 2023-12-14T18:36:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 33189, 2023-12-14T18:36:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 1, 2023-12-14T18:36:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 33713, 2023-12-14T18:36:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 33712, 2023-12-14T18:36:47.079 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 287620, 2023-12-14T18:36:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 198964 2023-12-14T18:36:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2023-12-14T18:36:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 60180, 2023-12-14T18:36:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 50, 2023-12-14T18:36:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 57917, 2023-12-14T18:36:47.080 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 18, 2023-12-14T18:36:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 118, 2023-12-14T18:36:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2023-12-14T18:36:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 8498, 2023-12-14T18:36:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 22.260728254, 2023-12-14T18:36:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.002619525 2023-12-14T18:36:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.081 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2023-12-14T18:36:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4, 2023-12-14T18:36:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.308663671, 2023-12-14T18:36:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.077165917 2023-12-14T18:36:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2023-12-14T18:36:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.082 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2023-12-14T18:36:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.083 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2023-12-14T18:36:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 19984, 2023-12-14T18:36:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 109.850201381, 2023-12-14T18:36:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.005496907 2023-12-14T18:36:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2023-12-14T18:36:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.084 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2023-12-14T18:36:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1, 2023-12-14T18:36:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000168964, 2023-12-14T18:36:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000168964 2023-12-14T18:36:47.085 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2023-12-14T18:36:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2023-12-14T18:36:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.086 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2023-12-14T18:36:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.087 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2023-12-14T18:36:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2023-12-14T18:36:47.088 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2023-12-14T18:36:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.089 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2023-12-14T18:36:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2023-12-14T18:36:47.090 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2023-12-14T18:36:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1317, 2023-12-14T18:36:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 15.575005499, 2023-12-14T18:36:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.011826124 2023-12-14T18:36:47.091 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2023-12-14T18:36:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2023-12-14T18:36:47.092 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2023-12-14T18:36:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.093 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2023-12-14T18:36:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2, 2023-12-14T18:36:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.080499791, 2023-12-14T18:36:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.040249895 2023-12-14T18:36:47.094 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2023-12-14T18:36:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2023-12-14T18:36:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.095 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2023-12-14T18:36:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.096 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2023-12-14T18:36:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2023-12-14T18:36:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.097 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2023-12-14T18:36:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.098 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2023-12-14T18:36:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2023-12-14T18:36:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.099 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2023-12-14T18:36:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 20456, 2023-12-14T18:36:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 143.868024247, 2023-12-14T18:36:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.007033047 2023-12-14T18:36:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.100 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:36:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:36:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2023-12-14T18:36:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:47.101 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2023-12-14T18:36:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 1, 2023-12-14T18:36:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 3, 2023-12-14T18:36:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 2, 2023-12-14T18:36:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 1, 2023-12-14T18:36:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2023-12-14T18:36:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 18563, 2023-12-14T18:36:47.102 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 18563, 2023-12-14T18:36:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 624 2023-12-14T18:36:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2023-12-14T18:36:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:47.103 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:47.104 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:47.105 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:47.106 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2023-12-14T18:36:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:47.107 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 1073369, 2023-12-14T18:36:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 22440, 2023-12-14T18:36:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2023-12-14T18:36:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2023-12-14T18:36:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:47.108 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 9200, 2023-12-14T18:36:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 175, 2023-12-14T18:36:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:47.109 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2023-12-14T18:36:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 102409350, 2023-12-14T18:36:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 903868, 2023-12-14T18:36:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2023-12-14T18:36:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2023-12-14T18:36:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2023-12-14T18:36:47.110 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2023-12-14T18:36:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2023-12-14T18:36:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 44511, 2023-12-14T18:36:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 392820039, 2023-12-14T18:36:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 33, 2023-12-14T18:36:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 44511, 2023-12-14T18:36:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2023-12-14T18:36:47.111 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 44511, 2023-12-14T18:36:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 370.465598895, 2023-12-14T18:36:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.008323012 2023-12-14T18:36:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2023-12-14T18:36:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2023-12-14T18:36:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 44511, 2023-12-14T18:36:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 45834 2023-12-14T18:36:47.112 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 44511, 2023-12-14T18:36:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 714, 2023-12-14T18:36:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 43797, 2023-12-14T18:36:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2023-12-14T18:36:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2023-12-14T18:36:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 0, 2023-12-14T18:36:47.113 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 10, 2023-12-14T18:36:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 352, 2023-12-14T18:36:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 1562, 2023-12-14T18:36:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 132, 2023-12-14T18:36:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2023-12-14T18:36:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2023-12-14T18:36:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2023-12-14T18:36:47.114 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 5, 2023-12-14T18:36:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 41169, 2023-12-14T18:36:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2023-12-14T18:36:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2023-12-14T18:36:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2023-12-14T18:36:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 0, 2023-12-14T18:36:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 20, 2023-12-14T18:36:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:36:47.115 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2023-12-14T18:36:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:36:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2023-12-14T18:36:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2023-12-14T18:36:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2023-12-14T18:36:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:36:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2023-12-14T18:36:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:36:47.116 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2023-12-14T18:36:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2023-12-14T18:36:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2023-12-14T18:36:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2023-12-14T18:36:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2023-12-14T18:36:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2023-12-14T18:36:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2023-12-14T18:36:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2023-12-14T18:36:47.117 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2023-12-14T18:36:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2023-12-14T18:36:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2023-12-14T18:36:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2023-12-14T18:36:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2023-12-14T18:36:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2023-12-14T18:36:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2023-12-14T18:36:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2023-12-14T18:36:47.118 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2023-12-14T18:36:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 96, 2023-12-14T18:36:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2023-12-14T18:36:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 12, 2023-12-14T18:36:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 12, 2023-12-14T18:36:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 12, 2023-12-14T18:36:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2023-12-14T18:36:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2023-12-14T18:36:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 1817, 2023-12-14T18:36:47.119 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 724, 2023-12-14T18:36:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 43 2023-12-14T18:36:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2023-12-14T18:36:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 1, 2023-12-14T18:36:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:36:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2023-12-14T18:36:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2023-12-14T18:36:47.120 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2023-12-14T18:36:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 58290 2023-12-14T18:36:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2023-12-14T18:36:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2023-12-14T18:36:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 16, 2023-12-14T18:36:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2023-12-14T18:36:47.121 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 8, 2023-12-14T18:36:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 20456, 2023-12-14T18:36:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2023-12-14T18:36:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-12-14T18:36:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 145154, 2023-12-14T18:36:47.122 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 119611791, 2023-12-14T18:36:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 145154, 2023-12-14T18:36:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 145154, 2023-12-14T18:36:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 119611791, 2023-12-14T18:36:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:47.123 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2023-12-14T18:36:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-12-14T18:36:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:47.124 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2023-12-14T18:36:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2023-12-14T18:36:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 44511, 2023-12-14T18:36:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 395394519, 2023-12-14T18:36:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 44511, 2023-12-14T18:36:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 395394519, 2023-12-14T18:36:47.125 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2023-12-14T18:36:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:47.126 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2023-12-14T18:36:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2023-12-14T18:36:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2023-12-14T18:36:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 44511, 2023-12-14T18:36:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 44511, 2023-12-14T18:36:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 44511, 2023-12-14T18:36:47.127 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 44511, 2023-12-14T18:36:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:36:47.128 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2023-12-14T18:36:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 20456, 2023-12-14T18:36:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 2580272, 2023-12-14T18:36:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 20456, 2023-12-14T18:36:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:47.129 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 352, 2023-12-14T18:36:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2580272, 2023-12-14T18:36:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:47.130 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x56309ea740a0": { 2023-12-14T18:36:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2023-12-14T18:36:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 76367, 2023-12-14T18:36:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 379008708, 2023-12-14T18:36:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 76367, 2023-12-14T18:36:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:47.131 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 1210, 2023-12-14T18:36:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 379008708, 2023-12-14T18:36:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:47.132 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:47.133 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-12-14T18:36:47.133 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:46 smithi097 ceph-mon[43486]: pgmap v321: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:47.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:46 smithi160 ceph-mon[43840]: pgmap v321: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:47.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:46 smithi093 ceph-mon[42070]: pgmap v321: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:47.400 INFO:tasks.check_counter:Daemon mds.e mds.exported=198 2023-12-14T18:36:47.400 INFO:tasks.check_counter:Daemon mds.e mds.imported=417 2023-12-14T18:36:47.400 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=806 2023-12-14T18:36:47.400 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=3065 2023-12-14T18:36:47.400 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2023-12-14T18:36:47.400 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:36:47.400 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:36:47.400 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=106 2023-12-14T18:36:47.401 DEBUG:tasks.check_counter:Getting stats from h 2023-12-14T18:36:47.401 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.h perf dump 2023-12-14T18:36:47.792 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:36:48.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:47 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:36:48.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:47 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:36:48.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:47 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2023-12-14T18:36:48.878 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-12-14T18:36:48.878 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:48.878 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1396, 2023-12-14T18:36:48.878 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1398, 2023-12-14T18:36:48.878 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 127528968, 2023-12-14T18:36:48.878 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 434166, 2023-12-14T18:36:48.878 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 4, 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.173903370, 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.041393824, 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.083828766, 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.036806139, 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1396, 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.073664986, 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000052768 2023-12-14T18:36:48.879 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.880 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:48.880 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.880 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.880 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.880 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:36:48.880 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:36:48.880 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.881 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:48.881 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 575, 2023-12-14T18:36:48.881 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 548, 2023-12-14T18:36:48.881 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 106839260, 2023-12-14T18:36:48.881 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 219731, 2023-12-14T18:36:48.881 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 6, 2023-12-14T18:36:48.881 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 5, 2023-12-14T18:36:48.881 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.118312247, 2023-12-14T18:36:48.881 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.017693750, 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.095097387, 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.011987929, 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 548, 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.045940184, 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000083832 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.882 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1504400, 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 117968 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 1061, 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 1212, 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 114215338, 2023-12-14T18:36:48.883 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 744303, 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 8, 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 6, 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.155172298, 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.024185853, 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.092473856, 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.030047994, 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 1209, 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.069444660, 2023-12-14T18:36:48.884 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000057439 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 415264 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.885 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 1, 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-MDSRank": { 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 4376, 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.682916531, 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000156059 2023-12-14T18:36:48.886 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: "finisher-PurgeQueue": { 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: "queue_len": 0, 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: "complete_latency": { 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.887 INFO:teuthology.orchestra.run.smithi097.stdout: "mds": { 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: "request": 0, 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: "reply": 0, 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: "reply_latency": { 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_reply": 0, 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: "forward": 0, 2023-12-14T18:36:48.888 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_complete": 0, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_fetch_keys": 0, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_commit": 0, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_split": 0, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_merge": 0, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes": 35923, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_top": 25116, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_bottom": 10796, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pin_tail": 11, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_pinned": 43, 2023-12-14T18:36:48.889 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_expired": 1049, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "inodes_with_caps": 0, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "caps": 0, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "subtrees": 3, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse": 0, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_hit": 0, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_forward": 0, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_discover": 0, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_remote_ino": 0, 2023-12-14T18:36:48.890 INFO:teuthology.orchestra.run.smithi097.stdout: "traverse_lock": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "load_cent": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "q": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "exported": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "exported_inodes": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "imported": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "imported_inodes": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_dir_fetch": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_backtrace_fetch": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "openino_peer_discover": 0, 2023-12-14T18:36:48.891 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rfiles": 7, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rbytes": 246, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "root_rsnaps": 6, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_fetch": 0, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_set_tag": 0, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_inodes": 0, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_dirfrag_rstats": 0, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "scrub_file_inodes": 0, 2023-12-14T18:36:48.892 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_inode_file_caps": 0, 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_revoke": 0, 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_grant": 0, 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps": 0, 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_caps_dirty": 0, 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_cap_release": 0, 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: "process_request_cap_release": 0 2023-12-14T18:36:48.893 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_cache": { 2023-12-14T18:36:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update": 0, 2023-12-14T18:36:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_update_receipt": 0, 2023-12-14T18:36:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_try_discover": 0, 2023-12-14T18:36:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_send_discover": 0, 2023-12-14T18:36:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: "dir_handle_discover": 0, 2023-12-14T18:36:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays": 0, 2023-12-14T18:36:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_delayed": 0, 2023-12-14T18:36:48.894 INFO:teuthology.orchestra.run.smithi097.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_created": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_enqueued": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_reintegrated": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "strays_migrated": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_processing": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_started": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "recovery_completed": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:36:48.895 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_exportdir": 0, 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_flush": 0, 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragmentdir": 0, 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_fragstats": 0, 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: "ireq_inodestats": 0 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_log": { 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: "evadd": 0, 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: "evex": 0, 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: "evtrm": 0, 2023-12-14T18:36:48.896 INFO:teuthology.orchestra.run.smithi097.stdout: "ev": 0, 2023-12-14T18:36:48.897 INFO:teuthology.orchestra.run.smithi097.stdout: "evexg": 0, 2023-12-14T18:36:48.897 INFO:teuthology.orchestra.run.smithi097.stdout: "evexd": 0, 2023-12-14T18:36:48.897 INFO:teuthology.orchestra.run.smithi097.stdout: "segadd": 0, 2023-12-14T18:36:48.897 INFO:teuthology.orchestra.run.smithi097.stdout: "segex": 0, 2023-12-14T18:36:48.897 INFO:teuthology.orchestra.run.smithi097.stdout: "segtrm": 0, 2023-12-14T18:36:48.897 INFO:teuthology.orchestra.run.smithi097.stdout: "seg": 2, 2023-12-14T18:36:48.897 INFO:teuthology.orchestra.run.smithi097.stdout: "segexg": 0, 2023-12-14T18:36:48.898 INFO:teuthology.orchestra.run.smithi097.stdout: "segexd": 0, 2023-12-14T18:36:48.898 INFO:teuthology.orchestra.run.smithi097.stdout: "expos": 348208657, 2023-12-14T18:36:48.898 INFO:teuthology.orchestra.run.smithi097.stdout: "wrpos": 0, 2023-12-14T18:36:48.898 INFO:teuthology.orchestra.run.smithi097.stdout: "rdpos": 348780095, 2023-12-14T18:36:48.898 INFO:teuthology.orchestra.run.smithi097.stdout: "jlat": { 2023-12-14T18:36:48.898 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.898 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.898 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: "replayed": 88798 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_mem": { 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: "ino": 13748, 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: "ino+": 57036, 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: "ino-": 43288, 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: "dir": 463, 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: "dir+": 1805, 2023-12-14T18:36:48.899 INFO:teuthology.orchestra.run.smithi097.stdout: "dir-": 1342, 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: "dn": 35923, 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: "dn+": 72555, 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: "dn-": 36632, 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: "cap": 0, 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: "cap+": 0, 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: "cap-": 0, 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: "rss": 217336, 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: "heap": 207156 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.900 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_server": { 2023-12-14T18:36:48.901 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_client_request": 0, 2023-12-14T18:36:48.901 INFO:teuthology.orchestra.run.smithi097.stdout: "dispatch_server_request": 0, 2023-12-14T18:36:48.901 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_request": 0, 2023-12-14T18:36:48.901 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_client_session": 0, 2023-12-14T18:36:48.901 INFO:teuthology.orchestra.run.smithi097.stdout: "handle_peer_request": 0, 2023-12-14T18:36:48.901 INFO:teuthology.orchestra.run.smithi097.stdout: "req_create_latency": { 2023-12-14T18:36:48.901 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.901 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.901 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getattr_latency": { 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getfilelock_latency": { 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.902 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.903 INFO:teuthology.orchestra.run.smithi097.stdout: "req_link_latency": { 2023-12-14T18:36:48.903 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.903 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.903 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.903 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.903 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookup_latency": { 2023-12-14T18:36:48.903 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.903 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.903 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookuphash_latency": { 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupino_latency": { 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.904 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.905 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupname_latency": { 2023-12-14T18:36:48.905 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.905 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.905 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.905 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.905 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupparent_latency": { 2023-12-14T18:36:48.905 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.905 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lookupsnap_latency": { 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: "req_lssnap_latency": { 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.906 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mkdir_latency": { 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mknod_latency": { 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.907 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.908 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.908 INFO:teuthology.orchestra.run.smithi097.stdout: "req_mksnap_latency": { 2023-12-14T18:36:48.908 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.908 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.908 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.908 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.908 INFO:teuthology.orchestra.run.smithi097.stdout: "req_open_latency": { 2023-12-14T18:36:48.908 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.909 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.909 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.909 INFO:teuthology.orchestra.run.smithi097.stdout: "req_readdir_latency": { 2023-12-14T18:36:48.909 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.909 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.909 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.909 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.909 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rename_latency": { 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: "req_renamesnap_latency": { 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.910 INFO:teuthology.orchestra.run.smithi097.stdout: "req_snapdiff_latency": { 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmdir_latency": { 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.911 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmsnap_latency": { 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: "req_rmxattr_latency": { 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.912 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setattr_latency": { 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setdirlayout_latency": { 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.913 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setfilelock_latency": { 2023-12-14T18:36:48.914 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.914 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.914 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.914 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setlayout_latency": { 2023-12-14T18:36:48.914 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.914 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.914 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.914 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.915 INFO:teuthology.orchestra.run.smithi097.stdout: "req_setxattr_latency": { 2023-12-14T18:36:48.915 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.915 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.915 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.915 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.915 INFO:teuthology.orchestra.run.smithi097.stdout: "req_symlink_latency": { 2023-12-14T18:36:48.915 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.915 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.915 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: "req_unlink_latency": { 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: "req_getvxattr_latency": { 2023-12-14T18:36:48.916 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.917 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.917 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.917 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:48.917 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.917 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_sessions": { 2023-12-14T18:36:48.917 INFO:teuthology.orchestra.run.smithi097.stdout: "session_count": 2, 2023-12-14T18:36:48.917 INFO:teuthology.orchestra.run.smithi097.stdout: "session_add": 6, 2023-12-14T18:36:48.917 INFO:teuthology.orchestra.run.smithi097.stdout: "session_remove": 4, 2023-12-14T18:36:48.917 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_open": 4, 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: "sessions_stale": 0, 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: "total_load": 0, 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: "average_load": 0, 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: "avg_session_uptime": 3205 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:48.918 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:48.919 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:48.920 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2023-12-14T18:36:48.921 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 706363, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 13832, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2023-12-14T18:36:48.922 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:48.923 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:48.923 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:48.923 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:48.923 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 9200, 2023-12-14T18:36:48.923 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 175, 2023-12-14T18:36:48.923 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:48.923 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:48.923 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:48.923 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2023-12-14T18:36:48.924 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 56914878, 2023-12-14T18:36:48.924 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 463325, 2023-12-14T18:36:48.924 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:48.924 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2023-12-14T18:36:48.924 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:48.924 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2023-12-14T18:36:48.924 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.924 INFO:teuthology.orchestra.run.smithi097.stdout: "objecter": { 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "op_active": 0, 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "op_laggy": 0, 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send": 2781, 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "op_send_bytes": 26, 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "op_resend": 4, 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "op_reply": 2781, 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "op_latency": { 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2781, 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 5.541904515, 2023-12-14T18:36:48.925 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.001992774 2023-12-14T18:36:48.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.926 INFO:teuthology.orchestra.run.smithi097.stdout: "op_inflight": 0, 2023-12-14T18:36:48.926 INFO:teuthology.orchestra.run.smithi097.stdout: "oplen_avg": { 2023-12-14T18:36:48.926 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 2781, 2023-12-14T18:36:48.926 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 2782 2023-12-14T18:36:48.926 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.926 INFO:teuthology.orchestra.run.smithi097.stdout: "op": 2781, 2023-12-14T18:36:48.926 INFO:teuthology.orchestra.run.smithi097.stdout: "op_r": 2781, 2023-12-14T18:36:48.926 INFO:teuthology.orchestra.run.smithi097.stdout: "op_w": 0, 2023-12-14T18:36:48.927 INFO:teuthology.orchestra.run.smithi097.stdout: "op_rmw": 0, 2023-12-14T18:36:48.927 INFO:teuthology.orchestra.run.smithi097.stdout: "op_pg": 0, 2023-12-14T18:36:48.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_stat": 1750, 2023-12-14T18:36:48.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_create": 0, 2023-12-14T18:36:48.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_read": 1028, 2023-12-14T18:36:48.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_write": 0, 2023-12-14T18:36:48.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writefull": 0, 2023-12-14T18:36:48.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_writesame": 0, 2023-12-14T18:36:48.927 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_append": 0, 2023-12-14T18:36:48.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_zero": 0, 2023-12-14T18:36:48.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_truncate": 0, 2023-12-14T18:36:48.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_delete": 0, 2023-12-14T18:36:48.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_mapext": 0, 2023-12-14T18:36:48.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_sparse_read": 0, 2023-12-14T18:36:48.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_clonerange": 0, 2023-12-14T18:36:48.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_getxattr": 2, 2023-12-14T18:36:48.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_setxattr": 0, 2023-12-14T18:36:48.928 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:36:48.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_rmxattr": 0, 2023-12-14T18:36:48.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:36:48.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_call": 0, 2023-12-14T18:36:48.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_watch": 0, 2023-12-14T18:36:48.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_notify": 0, 2023-12-14T18:36:48.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:36:48.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls": 0, 2023-12-14T18:36:48.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:36:48.929 INFO:teuthology.orchestra.run.smithi097.stdout: "osdop_other": 0, 2023-12-14T18:36:48.930 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_active": 0, 2023-12-14T18:36:48.930 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_send": 0, 2023-12-14T18:36:48.930 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_resend": 0, 2023-12-14T18:36:48.930 INFO:teuthology.orchestra.run.smithi097.stdout: "linger_ping": 0, 2023-12-14T18:36:48.930 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_active": 0, 2023-12-14T18:36:48.930 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_send": 0, 2023-12-14T18:36:48.930 INFO:teuthology.orchestra.run.smithi097.stdout: "poolop_resend": 0, 2023-12-14T18:36:48.930 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_active": 0, 2023-12-14T18:36:48.930 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_send": 0, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "poolstat_resend": 0, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_active": 0, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_send": 0, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "statfs_resend": 0, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "command_active": 0, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "command_send": 0, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "command_resend": 0, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "map_epoch": 96, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "map_full": 0, 2023-12-14T18:36:48.931 INFO:teuthology.orchestra.run.smithi097.stdout: "map_inc": 12, 2023-12-14T18:36:48.932 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_sessions": 11, 2023-12-14T18:36:48.932 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_open": 11, 2023-12-14T18:36:48.932 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_session_close": 0, 2023-12-14T18:36:48.932 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_laggy": 0, 2023-12-14T18:36:48.932 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_wr": 0, 2023-12-14T18:36:48.932 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_rd": 2, 2023-12-14T18:36:48.932 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_del": 0 2023-12-14T18:36:48.932 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.932 INFO:teuthology.orchestra.run.smithi097.stdout: "oft": { 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_objs": 0, 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_updates": 0, 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: "omap_total_removes": 0 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: "osd-slow-ops": { 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: "slow_ops_count": 0 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: "purge_queue": { 2023-12-14T18:36:48.933 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops": 0, 2023-12-14T18:36:48.934 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_ops_high_water": 0, 2023-12-14T18:36:48.934 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing": 0, 2023-12-14T18:36:48.934 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executing_high_water": 0, 2023-12-14T18:36:48.934 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_executed": 0, 2023-12-14T18:36:48.934 INFO:teuthology.orchestra.run.smithi097.stdout: "pq_item_in_journal": 0 2023-12-14T18:36:48.934 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.934 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:48.934 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:48.934 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 3028, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 348295589, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 3028, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 3028, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 348295589, 2023-12-14T18:36:48.935 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_bytes": { 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:48.936 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2023-12-14T18:36:48.937 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2023-12-14T18:36:48.937 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:48.937 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:48.937 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2781, 2023-12-14T18:36:48.937 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 346311516, 2023-12-14T18:36:48.937 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2781, 2023-12-14T18:36:48.937 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 346311516, 2023-12-14T18:36:48.937 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:48.937 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.938 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.938 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.938 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:48.938 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.938 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-objecter_ops": { 2023-12-14T18:36:48.938 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:48.938 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 1024, 2023-12-14T18:36:48.938 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:48.938 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 2781, 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 2781, 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 2781, 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 2781, 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.939 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2023-12-14T18:36:48.940 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:48.941 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:48.941 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:48.941 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:48.941 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2023-12-14T18:36:48.941 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2023-12-14T18:36:48.941 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:48.941 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.941 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.941 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-write_buf_throttle-0x56190229c460": { 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 3758096384, 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 0, 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 0, 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:48.942 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 0, 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 0, 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:48.943 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:48.944 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-12-14T18:36:49.272 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2023-12-14T18:36:49.273 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2023-12-14T18:36:49.273 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2023-12-14T18:36:49.273 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=0 2023-12-14T18:36:49.273 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2023-12-14T18:36:49.273 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:36:49.273 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:36:49.273 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2023-12-14T18:36:49.273 DEBUG:tasks.check_counter:Getting stats from k 2023-12-14T18:36:49.274 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.k perf dump 2023-12-14T18:36:49.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: pgmap v322: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:49.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi160 to 182.5M 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: Unable to set osd_memory_target on smithi160 to 191374131: error parsing value: Value '191374131' is below minimum 939524096 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi093 to 182.4M 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: Unable to set osd_memory_target on smithi093 to 191315353: error parsing value: Value '191315353' is below minimum 939524096 2023-12-14T18:36:49.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:36:49.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:36:49.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:49 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: pgmap v322: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:49.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi160 to 182.5M 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: Unable to set osd_memory_target on smithi160 to 191374131: error parsing value: Value '191374131' is below minimum 939524096 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi093 to 182.4M 2023-12-14T18:36:49.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: Unable to set osd_memory_target on smithi093 to 191315353: error parsing value: Value '191315353' is below minimum 939524096 2023-12-14T18:36:49.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:36:49.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:36:49.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:49 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.582 INFO:teuthology.orchestra.run.smithi097.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.b/config 2023-12-14T18:36:49.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: pgmap v322: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:49.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:49.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi160 to 182.5M 2023-12-14T18:36:49.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: Unable to set osd_memory_target on smithi160 to 191374131: error parsing value: Value '191374131' is below minimum 939524096 2023-12-14T18:36:49.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi093 to 182.4M 2023-12-14T18:36:49.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: Unable to set osd_memory_target on smithi093 to 191315353: error parsing value: Value '191315353' is below minimum 939524096 2023-12-14T18:36:49.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2023-12-14T18:36:49.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2023-12-14T18:36:49.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2023-12-14T18:36:49.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:49 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' 2023-12-14T18:36:50.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:50 smithi097 ceph-mon[43486]: Adjusting osd_memory_target on smithi097 to 182.5M 2023-12-14T18:36:50.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:50 smithi097 ceph-mon[43486]: Unable to set osd_memory_target on smithi097 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2023-12-14T18:36:50.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:50 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:36:50.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:50 smithi097 ceph-mon[43486]: Service mds.all has no related daemon already placed. 2023-12-14T18:36:50.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:50 smithi097 ceph-mon[43486]: Service mgr has no related daemon already placed. 2023-12-14T18:36:50.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:50 smithi097 ceph-mon[43486]: Service mon has no related daemon already placed. 2023-12-14T18:36:50.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:50 smithi160 ceph-mon[43840]: Adjusting osd_memory_target on smithi097 to 182.5M 2023-12-14T18:36:50.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:50 smithi160 ceph-mon[43840]: Unable to set osd_memory_target on smithi097 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2023-12-14T18:36:50.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:50 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:36:50.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:50 smithi160 ceph-mon[43840]: Service mds.all has no related daemon already placed. 2023-12-14T18:36:50.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:50 smithi160 ceph-mon[43840]: Service mgr has no related daemon already placed. 2023-12-14T18:36:50.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:50 smithi160 ceph-mon[43840]: Service mon has no related daemon already placed. 2023-12-14T18:36:50.552 INFO:teuthology.orchestra.run.smithi097.stdout:{ 2023-12-14T18:36:50.552 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:50.553 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 214, 2023-12-14T18:36:50.553 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 204, 2023-12-14T18:36:50.553 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 1419536, 2023-12-14T18:36:50.553 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 116192, 2023-12-14T18:36:50.553 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 4, 2023-12-14T18:36:50.553 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 0, 2023-12-14T18:36:50.553 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.040453654, 2023-12-14T18:36:50.553 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.011310683, 2023-12-14T18:36:50.553 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.031547397, 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.005175344, 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 203, 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.012668059, 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000062404 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:50.554 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:50.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.555 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 1419536, 2023-12-14T18:36:50.555 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 116192 2023-12-14T18:36:50.555 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.555 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:50.555 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 6, 2023-12-14T18:36:50.555 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 158, 2023-12-14T18:36:50.555 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 3312, 2023-12-14T18:36:50.555 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 179632, 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 5, 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 3, 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.009950581, 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.006054155, 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.001854342, 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 156, 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.019771169, 2023-12-14T18:36:50.556 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000126738 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 179632 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.557 INFO:teuthology.orchestra.run.smithi097.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:50.558 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_messages": 0, 2023-12-14T18:36:50.558 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages": 2, 2023-12-14T18:36:50.558 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_bytes": 0, 2023-12-14T18:36:50.558 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_bytes": 0, 2023-12-14T18:36:50.558 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_created_connections": 2, 2023-12-14T18:36:50.558 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_active_connections": 1, 2023-12-14T18:36:50.558 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_total_time": 0.000441513, 2023-12-14T18:36:50.558 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_send_time": 0.000000000, 2023-12-14T18:36:50.558 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_recv_time": 0.000360604, 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:50.559 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:50.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.560 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:36:50.560 INFO:teuthology.orchestra.run.smithi097.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:36:50.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.560 INFO:teuthology.orchestra.run.smithi097.stdout: "cct": { 2023-12-14T18:36:50.560 INFO:teuthology.orchestra.run.smithi097.stdout: "total_workers": 0, 2023-12-14T18:36:50.560 INFO:teuthology.orchestra.run.smithi097.stdout: "unhealthy_workers": 0 2023-12-14T18:36:50.560 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.560 INFO:teuthology.orchestra.run.smithi097.stdout: "mempool": { 2023-12-14T18:36:50.561 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:50.561 INFO:teuthology.orchestra.run.smithi097.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:50.561 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:50.561 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:50.561 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:50.561 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:50.561 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:50.561 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:50.561 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:50.562 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:50.563 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_items": 0, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_bytes": 149024, 2023-12-14T18:36:50.564 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_anon_items": 31, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_bytes": 0, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_items": 0, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_bytes": 0, 2023-12-14T18:36:50.565 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_items": 0, 2023-12-14T18:36:50.566 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:50.566 INFO:teuthology.orchestra.run.smithi097.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:50.566 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:50.566 INFO:teuthology.orchestra.run.smithi097.stdout: "pgmap_items": 0, 2023-12-14T18:36:50.566 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_bytes": 928, 2023-12-14T18:36:50.566 INFO:teuthology.orchestra.run.smithi097.stdout: "mds_co_items": 2, 2023-12-14T18:36:50.566 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:50.566 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_1_items": 0, 2023-12-14T18:36:50.566 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:50.567 INFO:teuthology.orchestra.run.smithi097.stdout: "unittest_2_items": 0 2023-12-14T18:36:50.567 INFO:teuthology.orchestra.run.smithi097.stdout: }, 2023-12-14T18:36:50.567 INFO:teuthology.orchestra.run.smithi097.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:50.567 INFO:teuthology.orchestra.run.smithi097.stdout: "val": 0, 2023-12-14T18:36:50.567 INFO:teuthology.orchestra.run.smithi097.stdout: "max": 104857600, 2023-12-14T18:36:50.567 INFO:teuthology.orchestra.run.smithi097.stdout: "get_started": 0, 2023-12-14T18:36:50.567 INFO:teuthology.orchestra.run.smithi097.stdout: "get": 216, 2023-12-14T18:36:50.567 INFO:teuthology.orchestra.run.smithi097.stdout: "get_sum": 1389337, 2023-12-14T18:36:50.567 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:50.568 INFO:teuthology.orchestra.run.smithi097.stdout: "get_or_fail_success": 216, 2023-12-14T18:36:50.568 INFO:teuthology.orchestra.run.smithi097.stdout: "take": 0, 2023-12-14T18:36:50.568 INFO:teuthology.orchestra.run.smithi097.stdout: "take_sum": 0, 2023-12-14T18:36:50.568 INFO:teuthology.orchestra.run.smithi097.stdout: "put": 216, 2023-12-14T18:36:50.568 INFO:teuthology.orchestra.run.smithi097.stdout: "put_sum": 1389337, 2023-12-14T18:36:50.568 INFO:teuthology.orchestra.run.smithi097.stdout: "wait": { 2023-12-14T18:36:50.568 INFO:teuthology.orchestra.run.smithi097.stdout: "avgcount": 0, 2023-12-14T18:36:50.568 INFO:teuthology.orchestra.run.smithi097.stdout: "sum": 0.000000000, 2023-12-14T18:36:50.568 INFO:teuthology.orchestra.run.smithi097.stdout: "avgtime": 0.000000000 2023-12-14T18:36:50.569 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:50.569 INFO:teuthology.orchestra.run.smithi097.stdout: } 2023-12-14T18:36:50.569 INFO:teuthology.orchestra.run.smithi097.stdout:} 2023-12-14T18:36:50.569 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:50 smithi093 ceph-mon[42070]: Adjusting osd_memory_target on smithi097 to 182.5M 2023-12-14T18:36:50.569 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:50 smithi093 ceph-mon[42070]: Unable to set osd_memory_target on smithi097 to 191371264: error parsing value: Value '191371264' is below minimum 939524096 2023-12-14T18:36:50.569 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:50 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:36:50.570 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:50 smithi093 ceph-mon[42070]: Service mds.all has no related daemon already placed. 2023-12-14T18:36:50.570 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:50 smithi093 ceph-mon[42070]: Service mgr has no related daemon already placed. 2023-12-14T18:36:50.570 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:50 smithi093 ceph-mon[42070]: Service mon has no related daemon already placed. 2023-12-14T18:36:50.913 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.k 2023-12-14T18:36:50.913 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.k 2023-12-14T18:36:50.913 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.k 2023-12-14T18:36:50.914 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.k 2023-12-14T18:36:50.914 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2023-12-14T18:36:50.914 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2023-12-14T18:36:50.914 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2023-12-14T18:36:50.914 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.k 2023-12-14T18:36:50.914 DEBUG:tasks.check_counter:Getting stats from c 2023-12-14T18:36:50.914 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.c perf dump 2023-12-14T18:36:51.246 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:51 smithi160 ceph-mon[43840]: pgmap v323: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:51.248 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:36:51.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:51 smithi093 ceph-mon[42070]: pgmap v323: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:51.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:51 smithi097 ceph-mon[43486]: pgmap v323: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:36:52.239 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2023-12-14T18:36:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 2115, 2023-12-14T18:36:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 2116, 2023-12-14T18:36:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 142755087, 2023-12-14T18:36:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 657461, 2023-12-14T18:36:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 8, 2023-12-14T18:36:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2023-12-14T18:36:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.231157993, 2023-12-14T18:36:52.240 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.046605486, 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.108524357, 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.061610466, 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2114, 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.094495192, 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000044699 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.241 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.242 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.242 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.242 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:36:52.242 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:36:52.242 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.242 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:52.242 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 548, 2023-12-14T18:36:52.242 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 521, 2023-12-14T18:36:52.242 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 102265085, 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 211762, 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 6, 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.107086944, 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.017173603, 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.084327104, 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.011571669, 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 521, 2023-12-14T18:36:52.243 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.027429099, 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000052647 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1504512, 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 118400 2023-12-14T18:36:52.244 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.245 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:52.245 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 407, 2023-12-14T18:36:52.245 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 559, 2023-12-14T18:36:52.245 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 138712994, 2023-12-14T18:36:52.245 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 541219, 2023-12-14T18:36:52.245 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 8, 2023-12-14T18:36:52.245 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 6, 2023-12-14T18:36:52.245 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.096054434, 2023-12-14T18:36:52.245 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.012565534, 2023-12-14T18:36:52.246 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.070149658, 2023-12-14T18:36:52.246 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.008929643, 2023-12-14T18:36:52.246 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:52.246 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 556, 2023-12-14T18:36:52.246 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.037423068, 2023-12-14T18:36:52.246 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000067307 2023-12-14T18:36:52.246 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.246 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:52.246 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 320, 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 415264 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2023-12-14T18:36:52.247 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.248 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2023-12-14T18:36:52.248 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2023-12-14T18:36:52.248 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2023-12-14T18:36:52.248 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 4331, 2023-12-14T18:36:52.248 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.111961865, 2023-12-14T18:36:52.248 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000025851 2023-12-14T18:36:52.248 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:52.248 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.248 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 0, 2023-12-14T18:36:52.249 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 0, 2023-12-14T18:36:52.250 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2023-12-14T18:36:52.250 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.250 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.250 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.250 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.250 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2023-12-14T18:36:52.250 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 0, 2023-12-14T18:36:52.250 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2023-12-14T18:36:52.250 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 0, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 65503, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 3663, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 40060, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 21780, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 60269, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 0, 2023-12-14T18:36:52.251 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 0, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 0, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 3, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 0, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 0, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 0, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2023-12-14T18:36:52.252 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 0, 2023-12-14T18:36:52.253 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2023-12-14T18:36:52.254 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 0, 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 0, 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 0, 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 0, 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 0, 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 0 2023-12-14T18:36:52.255 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.256 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2023-12-14T18:36:52.256 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2023-12-14T18:36:52.256 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2023-12-14T18:36:52.256 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2023-12-14T18:36:52.256 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 0, 2023-12-14T18:36:52.256 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 0, 2023-12-14T18:36:52.256 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2023-12-14T18:36:52.256 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2023-12-14T18:36:52.256 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2023-12-14T18:36:52.257 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 0, 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2023-12-14T18:36:52.258 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2023-12-14T18:36:52.259 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 0, 2023-12-14T18:36:52.259 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2023-12-14T18:36:52.259 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2023-12-14T18:36:52.259 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 0, 2023-12-14T18:36:52.259 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2023-12-14T18:36:52.259 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2023-12-14T18:36:52.259 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 122, 2023-12-14T18:36:52.259 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 0, 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 384752268, 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.260 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 90673 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 29024, 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 62249, 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 33225, 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 1033, 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 1410, 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 377, 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 65503, 2023-12-14T18:36:52.261 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 82712, 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 17209, 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 0, 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 0, 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 0, 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 231336, 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 207156 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 0, 2023-12-14T18:36:52.262 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 0, 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 0, 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 0, 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2023-12-14T18:36:52.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2023-12-14T18:36:52.264 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.265 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.265 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2023-12-14T18:36:52.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.265 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.266 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.267 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2023-12-14T18:36:52.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.267 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.267 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.267 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2023-12-14T18:36:52.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.267 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.268 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2023-12-14T18:36:52.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.268 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.268 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2023-12-14T18:36:52.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.268 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.269 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.269 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2023-12-14T18:36:52.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.269 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.269 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.269 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.269 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2023-12-14T18:36:52.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.270 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.270 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.270 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2023-12-14T18:36:52.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.270 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.270 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.270 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.271 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2023-12-14T18:36:52.271 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.271 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.271 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.271 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.271 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2023-12-14T18:36:52.271 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.271 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.271 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.272 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.272 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2023-12-14T18:36:52.272 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.272 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.272 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.272 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.272 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2023-12-14T18:36:52.272 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.272 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.273 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.273 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.273 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2023-12-14T18:36:52.273 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.273 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.273 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.273 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.274 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2023-12-14T18:36:52.274 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.274 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.274 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.275 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.275 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2023-12-14T18:36:52.275 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.275 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.275 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.275 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.276 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2023-12-14T18:36:52.276 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.276 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.276 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.276 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.276 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2023-12-14T18:36:52.276 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.276 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.276 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.277 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.277 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2023-12-14T18:36:52.277 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.277 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.277 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.277 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.277 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2023-12-14T18:36:52.278 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.278 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.278 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.278 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.278 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2023-12-14T18:36:52.278 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.278 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.278 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.278 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.279 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2023-12-14T18:36:52.279 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.279 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.279 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.279 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.279 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2023-12-14T18:36:52.279 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.279 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.279 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.280 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.280 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2023-12-14T18:36:52.280 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.280 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.280 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.280 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.280 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:36:52.280 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:36:52.280 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2023-12-14T18:36:52.281 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.281 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.281 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.281 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:52.281 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.281 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2023-12-14T18:36:52.281 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 2, 2023-12-14T18:36:52.281 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 4, 2023-12-14T18:36:52.282 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 2, 2023-12-14T18:36:52.282 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 3, 2023-12-14T18:36:52.282 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2023-12-14T18:36:52.282 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 0, 2023-12-14T18:36:52.282 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 0, 2023-12-14T18:36:52.282 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 2481 2023-12-14T18:36:52.282 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.282 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2023-12-14T18:36:52.282 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:52.283 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:52.284 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:52.285 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:52.285 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:52.285 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:52.285 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:52.285 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:52.285 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:52.285 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:52.285 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:52.285 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:52.286 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:52.286 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2023-12-14T18:36:52.286 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:52.286 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:52.286 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:52.286 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:52.286 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 1225213, 2023-12-14T18:36:52.286 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 29094, 2023-12-14T18:36:52.286 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 9200, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 175, 2023-12-14T18:36:52.287 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:52.288 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:52.288 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:52.288 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2023-12-14T18:36:52.288 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 112736697, 2023-12-14T18:36:52.288 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 915431, 2023-12-14T18:36:52.288 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:52.288 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2023-12-14T18:36:52.288 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:52.288 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2023-12-14T18:36:52.289 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.289 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2023-12-14T18:36:52.289 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2023-12-14T18:36:52.289 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2023-12-14T18:36:52.289 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 2819, 2023-12-14T18:36:52.289 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 21, 2023-12-14T18:36:52.289 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 3, 2023-12-14T18:36:52.289 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 2819, 2023-12-14T18:36:52.289 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2023-12-14T18:36:52.290 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2819, 2023-12-14T18:36:52.290 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 4.822925581, 2023-12-14T18:36:52.290 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001710863 2023-12-14T18:36:52.290 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.290 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2023-12-14T18:36:52.290 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2023-12-14T18:36:52.290 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2819, 2023-12-14T18:36:52.290 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2820 2023-12-14T18:36:52.290 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.291 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 2819, 2023-12-14T18:36:52.291 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 2819, 2023-12-14T18:36:52.291 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 0, 2023-12-14T18:36:52.291 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2023-12-14T18:36:52.291 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2023-12-14T18:36:52.291 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 1831, 2023-12-14T18:36:52.291 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 0, 2023-12-14T18:36:52.291 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 986, 2023-12-14T18:36:52.291 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2023-12-14T18:36:52.292 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 1, 2023-12-14T18:36:52.293 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 0, 2023-12-14T18:36:52.293 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:36:52.293 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2023-12-14T18:36:52.293 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:36:52.293 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2023-12-14T18:36:52.293 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2023-12-14T18:36:52.293 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2023-12-14T18:36:52.293 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2023-12-14T18:36:52.294 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 96, 2023-12-14T18:36:52.295 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2023-12-14T18:36:52.296 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 12, 2023-12-14T18:36:52.296 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 11, 2023-12-14T18:36:52.296 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 11, 2023-12-14T18:36:52.296 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2023-12-14T18:36:52.296 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2023-12-14T18:36:52.296 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 0, 2023-12-14T18:36:52.296 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 2, 2023-12-14T18:36:52.296 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 0 2023-12-14T18:36:52.296 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.297 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2023-12-14T18:36:52.297 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 0, 2023-12-14T18:36:52.297 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:36:52.297 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2023-12-14T18:36:52.297 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2023-12-14T18:36:52.297 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.297 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2023-12-14T18:36:52.297 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.298 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 3066, 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 383441763, 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 3066, 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:52.299 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 3066, 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 383441763, 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2023-12-14T18:36:52.300 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:52.301 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:52.301 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:52.301 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:52.301 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:52.301 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 2819, 2023-12-14T18:36:52.301 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 381449588, 2023-12-14T18:36:52.301 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2819, 2023-12-14T18:36:52.301 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 381449588, 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2023-12-14T18:36:52.302 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 2819, 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 2819, 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2819, 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 2819, 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:52.303 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:52.304 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:52.305 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:52.305 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:52.305 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:52.305 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:52.305 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2023-12-14T18:36:52.305 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2023-12-14T18:36:52.305 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:52.305 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.305 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x5571f0356460": { 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:52.306 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:52.307 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:52.308 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:52.308 INFO:teuthology.orchestra.run.smithi160.stdout:} 2023-12-14T18:36:52.618 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2023-12-14T18:36:52.618 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2023-12-14T18:36:52.618 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2023-12-14T18:36:52.618 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2023-12-14T18:36:52.618 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=0 2023-12-14T18:36:52.618 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:36:52.618 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:36:52.619 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2023-12-14T18:36:52.619 DEBUG:tasks.check_counter:Getting stats from f 2023-12-14T18:36:52.619 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.f perf dump 2023-12-14T18:36:52.929 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:36:53.396 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:53 smithi160 ceph-mon[43840]: pgmap v324: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-12-14T18:36:53.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:53 smithi093 ceph-mon[42070]: pgmap v324: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-12-14T18:36:53.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:53 smithi097 ceph-mon[43486]: pgmap v324: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 KiB/s rd, 5 op/s 2023-12-14T18:36:53.929 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2023-12-14T18:36:53.929 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:53.930 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 699, 2023-12-14T18:36:53.930 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 701, 2023-12-14T18:36:53.930 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 147055227, 2023-12-14T18:36:53.930 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 217453, 2023-12-14T18:36:53.930 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 8, 2023-12-14T18:36:53.930 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2023-12-14T18:36:53.930 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.122842213, 2023-12-14T18:36:53.930 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.014043378, 2023-12-14T18:36:53.931 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.085067197, 2023-12-14T18:36:53.931 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.017861011, 2023-12-14T18:36:53.931 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:53.931 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 699, 2023-12-14T18:36:53.931 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.035227113, 2023-12-14T18:36:53.931 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000050396 2023-12-14T18:36:53.931 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.931 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:53.932 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.932 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.932 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.932 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.932 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 0, 2023-12-14T18:36:53.932 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 0 2023-12-14T18:36:53.932 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.932 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:53.932 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 893, 2023-12-14T18:36:53.933 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1044, 2023-12-14T18:36:53.933 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 152175720, 2023-12-14T18:36:53.933 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 691750, 2023-12-14T18:36:53.933 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 9, 2023-12-14T18:36:53.933 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 7, 2023-12-14T18:36:53.933 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.141309573, 2023-12-14T18:36:53.933 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.026848528, 2023-12-14T18:36:53.933 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.080866773, 2023-12-14T18:36:53.934 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.025526623, 2023-12-14T18:36:53.934 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:53.934 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1042, 2023-12-14T18:36:53.934 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.057241547, 2023-12-14T18:36:53.934 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000054934 2023-12-14T18:36:53.934 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.934 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:53.934 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.935 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.935 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.935 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.935 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3312, 2023-12-14T18:36:53.935 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 415888 2023-12-14T18:36:53.935 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.935 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:53.935 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 1454, 2023-12-14T18:36:53.936 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 1432, 2023-12-14T18:36:53.936 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 83123713, 2023-12-14T18:36:53.936 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 495476, 2023-12-14T18:36:53.936 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 5, 2023-12-14T18:36:53.936 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 4, 2023-12-14T18:36:53.936 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.192183803, 2023-12-14T18:36:53.936 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.045155223, 2023-12-14T18:36:53.936 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.103101111, 2023-12-14T18:36:53.936 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.042846015, 2023-12-14T18:36:53.937 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:53.937 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1431, 2023-12-14T18:36:53.937 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.086308986, 2023-12-14T18:36:53.937 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000060313 2023-12-14T18:36:53.937 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.937 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:53.937 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.937 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.938 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.938 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1492208, 2023-12-14T18:36:53.938 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 118544 2023-12-14T18:36:53.938 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.938 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2023-12-14T18:36:53.938 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2023-12-14T18:36:53.939 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2023-12-14T18:36:53.939 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.939 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2023-12-14T18:36:53.939 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2023-12-14T18:36:53.939 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2023-12-14T18:36:53.939 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 4268, 2023-12-14T18:36:53.939 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.115181033, 2023-12-14T18:36:53.939 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000026987 2023-12-14T18:36:53.940 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:53.940 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.940 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2023-12-14T18:36:53.940 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2023-12-14T18:36:53.940 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2023-12-14T18:36:53.940 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.940 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.940 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.940 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:53.941 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.941 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2023-12-14T18:36:53.941 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 0, 2023-12-14T18:36:53.941 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 0, 2023-12-14T18:36:53.941 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2023-12-14T18:36:53.941 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.941 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.941 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.942 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.942 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2023-12-14T18:36:53.942 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 0, 2023-12-14T18:36:53.942 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2023-12-14T18:36:53.942 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2023-12-14T18:36:53.942 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 0, 2023-12-14T18:36:53.942 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 0, 2023-12-14T18:36:53.942 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 0, 2023-12-14T18:36:53.942 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 70745, 2023-12-14T18:36:53.943 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 2554, 2023-12-14T18:36:53.943 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 51150, 2023-12-14T18:36:53.943 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 17041, 2023-12-14T18:36:53.943 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 67096, 2023-12-14T18:36:53.943 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 0, 2023-12-14T18:36:53.943 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 0, 2023-12-14T18:36:53.943 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 0, 2023-12-14T18:36:53.944 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 1, 2023-12-14T18:36:53.944 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 0, 2023-12-14T18:36:53.944 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 0, 2023-12-14T18:36:53.944 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 0, 2023-12-14T18:36:53.944 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 0, 2023-12-14T18:36:53.944 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:36:53.944 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2023-12-14T18:36:53.944 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2023-12-14T18:36:53.944 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 0, 2023-12-14T18:36:53.945 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 0, 2023-12-14T18:36:53.945 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 0, 2023-12-14T18:36:53.945 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 0, 2023-12-14T18:36:53.945 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 0, 2023-12-14T18:36:53.945 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2023-12-14T18:36:53.945 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2023-12-14T18:36:53.945 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2023-12-14T18:36:53.945 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2023-12-14T18:36:53.946 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 0, 2023-12-14T18:36:53.946 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 0, 2023-12-14T18:36:53.946 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 0, 2023-12-14T18:36:53.946 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2023-12-14T18:36:53.946 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2023-12-14T18:36:53.946 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:36:53.946 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:36:53.946 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2023-12-14T18:36:53.947 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:36:53.947 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2023-12-14T18:36:53.947 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2023-12-14T18:36:53.947 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 0, 2023-12-14T18:36:53.947 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 0, 2023-12-14T18:36:53.947 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 0, 2023-12-14T18:36:53.947 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:36:53.947 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 0, 2023-12-14T18:36:53.947 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:36:53.948 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 0, 2023-12-14T18:36:53.948 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 0, 2023-12-14T18:36:53.948 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 0, 2023-12-14T18:36:53.948 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 0 2023-12-14T18:36:53.948 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.948 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2023-12-14T18:36:53.948 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 0, 2023-12-14T18:36:53.948 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 0, 2023-12-14T18:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 0, 2023-12-14T18:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 0, 2023-12-14T18:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 0, 2023-12-14T18:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 0, 2023-12-14T18:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2023-12-14T18:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 0, 2023-12-14T18:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 0, 2023-12-14T18:36:53.949 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2023-12-14T18:36:53.950 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2023-12-14T18:36:53.950 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2023-12-14T18:36:53.950 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:36:53.950 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:36:53.950 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2023-12-14T18:36:53.950 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2023-12-14T18:36:53.950 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:36:53.950 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 0, 2023-12-14T18:36:53.951 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2023-12-14T18:36:53.951 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 0, 2023-12-14T18:36:53.951 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2023-12-14T18:36:53.951 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2023-12-14T18:36:53.951 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.951 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2023-12-14T18:36:53.951 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 0, 2023-12-14T18:36:53.951 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2023-12-14T18:36:53.951 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2023-12-14T18:36:53.952 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 0, 2023-12-14T18:36:53.952 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2023-12-14T18:36:53.952 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2023-12-14T18:36:53.952 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 0, 2023-12-14T18:36:53.952 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2023-12-14T18:36:53.952 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2023-12-14T18:36:53.952 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 108, 2023-12-14T18:36:53.952 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2023-12-14T18:36:53.952 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2023-12-14T18:36:53.953 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2023-12-14T18:36:53.953 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 0, 2023-12-14T18:36:53.953 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 383199978, 2023-12-14T18:36:53.953 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2023-12-14T18:36:53.953 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.953 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.953 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.953 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.954 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 76367 2023-12-14T18:36:53.954 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.954 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2023-12-14T18:36:53.954 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 30290, 2023-12-14T18:36:53.954 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 58232, 2023-12-14T18:36:53.954 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 27942, 2023-12-14T18:36:53.954 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 1028, 2023-12-14T18:36:53.954 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 1324, 2023-12-14T18:36:53.954 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 296, 2023-12-14T18:36:53.955 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 70745, 2023-12-14T18:36:53.955 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 81966, 2023-12-14T18:36:53.955 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 11221, 2023-12-14T18:36:53.955 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 0, 2023-12-14T18:36:53.955 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 0, 2023-12-14T18:36:53.955 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 0, 2023-12-14T18:36:53.955 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 245028, 2023-12-14T18:36:53.955 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 207156 2023-12-14T18:36:53.956 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.956 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2023-12-14T18:36:53.956 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 0, 2023-12-14T18:36:53.956 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 0, 2023-12-14T18:36:53.956 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 0, 2023-12-14T18:36:53.956 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 0, 2023-12-14T18:36:53.956 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 0, 2023-12-14T18:36:53.956 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2023-12-14T18:36:53.956 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.957 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.957 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.957 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.957 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2023-12-14T18:36:53.957 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.957 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.957 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.958 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.958 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2023-12-14T18:36:53.958 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.958 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.958 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.958 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.958 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2023-12-14T18:36:53.958 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.958 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.959 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.959 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.959 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2023-12-14T18:36:53.959 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.959 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.959 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.959 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.959 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2023-12-14T18:36:53.959 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.960 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.960 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.960 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.960 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2023-12-14T18:36:53.960 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.960 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.960 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.960 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.961 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2023-12-14T18:36:53.961 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.961 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.961 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.961 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.961 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2023-12-14T18:36:53.961 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.961 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.961 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.962 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.962 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2023-12-14T18:36:53.962 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.962 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.962 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.962 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.962 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2023-12-14T18:36:53.962 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.962 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.963 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.963 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.963 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2023-12-14T18:36:53.963 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.963 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.963 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.963 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.964 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2023-12-14T18:36:53.964 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.964 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.964 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.964 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.964 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2023-12-14T18:36:53.964 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.964 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.965 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.965 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.965 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2023-12-14T18:36:53.965 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.965 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.965 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.965 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.965 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2023-12-14T18:36:53.966 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.966 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.966 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.966 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.966 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2023-12-14T18:36:53.966 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.966 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.966 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.966 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.967 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2023-12-14T18:36:53.967 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.967 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.967 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.967 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.967 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2023-12-14T18:36:53.967 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.967 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.967 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.968 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.968 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2023-12-14T18:36:53.968 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.968 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.968 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.968 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.968 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2023-12-14T18:36:53.968 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.968 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.969 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.969 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.969 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2023-12-14T18:36:53.969 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.969 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.969 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.969 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.969 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2023-12-14T18:36:53.969 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.970 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.970 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.970 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.970 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2023-12-14T18:36:53.970 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.970 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.970 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.970 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.970 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2023-12-14T18:36:53.971 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.971 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.971 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.971 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.971 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2023-12-14T18:36:53.971 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.971 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.971 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.971 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.972 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2023-12-14T18:36:53.972 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.972 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.972 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.972 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.972 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2023-12-14T18:36:53.972 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.972 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.972 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.973 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.973 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2023-12-14T18:36:53.973 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.973 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.973 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.973 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.973 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:36:53.973 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:36:53.974 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2023-12-14T18:36:53.974 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.974 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.974 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.974 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:53.974 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.974 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2023-12-14T18:36:53.974 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 1, 2023-12-14T18:36:53.974 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 3, 2023-12-14T18:36:53.975 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 2, 2023-12-14T18:36:53.975 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 3, 2023-12-14T18:36:53.975 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2023-12-14T18:36:53.975 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 0, 2023-12-14T18:36:53.975 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 0, 2023-12-14T18:36:53.975 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 4348 2023-12-14T18:36:53.975 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.975 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2023-12-14T18:36:53.975 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:53.976 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:53.976 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:53.976 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:53.976 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:53.976 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:53.976 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:53.976 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:53.976 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:53.976 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:53.977 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:53.977 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:53.977 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:53.977 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:53.977 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:53.977 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:53.977 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:53.977 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:53.978 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:53.978 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:53.978 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:53.978 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:53.978 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:53.978 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:53.978 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:53.978 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:53.978 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:53.979 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:53.979 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:53.979 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:53.979 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:53.979 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2023-12-14T18:36:53.979 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:53.979 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:53.979 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:53.979 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:53.980 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 1269121, 2023-12-14T18:36:53.980 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 30363, 2023-12-14T18:36:53.980 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:53.980 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:53.980 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2023-12-14T18:36:53.980 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2023-12-14T18:36:53.980 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:53.980 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:53.980 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:53.981 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:53.981 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 9200, 2023-12-14T18:36:53.981 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 175, 2023-12-14T18:36:53.981 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:53.981 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:53.981 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:53.981 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2023-12-14T18:36:53.981 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 119287532, 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 968100, 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2023-12-14T18:36:53.982 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 2798, 2023-12-14T18:36:53.983 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 21, 2023-12-14T18:36:53.983 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 3, 2023-12-14T18:36:53.983 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 2798, 2023-12-14T18:36:53.983 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2023-12-14T18:36:53.983 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2798, 2023-12-14T18:36:53.983 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 3.971938678, 2023-12-14T18:36:53.983 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.001419563 2023-12-14T18:36:53.983 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.983 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2023-12-14T18:36:53.984 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2023-12-14T18:36:53.984 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2798, 2023-12-14T18:36:53.984 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2799 2023-12-14T18:36:53.984 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.984 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 2798, 2023-12-14T18:36:53.984 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 2798, 2023-12-14T18:36:53.984 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 0, 2023-12-14T18:36:53.984 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2023-12-14T18:36:53.984 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2023-12-14T18:36:53.985 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 1832, 2023-12-14T18:36:53.985 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 0, 2023-12-14T18:36:53.985 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 964, 2023-12-14T18:36:53.985 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 0, 2023-12-14T18:36:53.985 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 0, 2023-12-14T18:36:53.985 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2023-12-14T18:36:53.985 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2023-12-14T18:36:53.985 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2023-12-14T18:36:53.985 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 0, 2023-12-14T18:36:53.986 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 0, 2023-12-14T18:36:53.986 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2023-12-14T18:36:53.986 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2023-12-14T18:36:53.986 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2023-12-14T18:36:53.986 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 1, 2023-12-14T18:36:53.986 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 0, 2023-12-14T18:36:53.986 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:36:53.986 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2023-12-14T18:36:53.986 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:36:53.987 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2023-12-14T18:36:53.987 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2023-12-14T18:36:53.987 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2023-12-14T18:36:53.987 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:36:53.987 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2023-12-14T18:36:53.987 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:36:53.987 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2023-12-14T18:36:53.987 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2023-12-14T18:36:53.987 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2023-12-14T18:36:53.988 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2023-12-14T18:36:53.988 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2023-12-14T18:36:53.988 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2023-12-14T18:36:53.988 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2023-12-14T18:36:53.988 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2023-12-14T18:36:53.988 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2023-12-14T18:36:53.988 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2023-12-14T18:36:53.988 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2023-12-14T18:36:53.988 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2023-12-14T18:36:53.989 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2023-12-14T18:36:53.989 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2023-12-14T18:36:53.989 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2023-12-14T18:36:53.989 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2023-12-14T18:36:53.989 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2023-12-14T18:36:53.989 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 96, 2023-12-14T18:36:53.989 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2023-12-14T18:36:53.989 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 12, 2023-12-14T18:36:53.989 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 11, 2023-12-14T18:36:53.990 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 11, 2023-12-14T18:36:53.990 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2023-12-14T18:36:53.990 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2023-12-14T18:36:53.990 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 0, 2023-12-14T18:36:53.990 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 2, 2023-12-14T18:36:53.990 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 0 2023-12-14T18:36:53.990 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.990 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2023-12-14T18:36:53.990 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 0, 2023-12-14T18:36:53.991 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:36:53.991 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2023-12-14T18:36:53.991 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2023-12-14T18:36:53.991 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.991 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2023-12-14T18:36:53.991 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 0 2023-12-14T18:36:53.991 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.991 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2023-12-14T18:36:53.991 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2023-12-14T18:36:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 0, 2023-12-14T18:36:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2023-12-14T18:36:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 0, 2023-12-14T18:36:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 0, 2023-12-14T18:36:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2023-12-14T18:36:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:53.992 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2023-12-14T18:36:53.993 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:53.993 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 3042, 2023-12-14T18:36:53.993 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 382065504, 2023-12-14T18:36:53.993 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:53.993 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 3042, 2023-12-14T18:36:53.993 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:53.993 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:53.993 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 3042, 2023-12-14T18:36:53.993 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 382065504, 2023-12-14T18:36:53.994 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:53.994 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.994 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.994 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.994 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:53.994 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.994 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2023-12-14T18:36:53.994 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:53.994 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 2798, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 380086312, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2798, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 380086312, 2023-12-14T18:36:53.995 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:53.996 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.996 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.996 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.996 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:53.996 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.996 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2023-12-14T18:36:53.996 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:53.996 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2023-12-14T18:36:53.996 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:53.997 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:53.997 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:53.997 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:53.997 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:53.997 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 2798, 2023-12-14T18:36:53.997 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 2798, 2023-12-14T18:36:53.997 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 2798, 2023-12-14T18:36:53.997 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 2798, 2023-12-14T18:36:53.997 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:53.998 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:53.998 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:53.998 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:53.998 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:53.998 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:53.998 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:36:53.998 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:53.998 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2023-12-14T18:36:53.998 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:53.999 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:54.000 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:54.000 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:54.000 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:54.000 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:54.000 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x55a7c1c5e460": { 2023-12-14T18:36:54.000 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:54.000 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2023-12-14T18:36:54.000 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:54.000 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:54.001 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:54.001 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:54.001 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:54.001 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:54.001 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:54.001 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 0, 2023-12-14T18:36:54.001 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 0, 2023-12-14T18:36:54.001 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:54.001 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:54.002 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:54.002 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:54.002 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:54.002 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:54.002 INFO:teuthology.orchestra.run.smithi160.stdout:} 2023-12-14T18:36:54.299 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2023-12-14T18:36:54.299 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2023-12-14T18:36:54.299 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2023-12-14T18:36:54.299 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2023-12-14T18:36:54.299 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2023-12-14T18:36:54.300 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:36:54.300 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:36:54.300 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2023-12-14T18:36:54.300 DEBUG:tasks.check_counter:Getting stats from i 2023-12-14T18:36:54.300 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.i perf dump 2023-12-14T18:36:54.611 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:36:55.396 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:55 smithi160 ceph-mon[43840]: pgmap v325: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:55.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:55 smithi093 ceph-mon[42070]: pgmap v325: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:55.588 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2023-12-14T18:36:55.588 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:55.589 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 125451, 2023-12-14T18:36:55.589 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 220620, 2023-12-14T18:36:55.589 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 102715367, 2023-12-14T18:36:55.589 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 311840587, 2023-12-14T18:36:55.589 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 11, 2023-12-14T18:36:55.590 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 5, 2023-12-14T18:36:55.590 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 7.007174825, 2023-12-14T18:36:55.590 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 3.142310535, 2023-12-14T18:36:55.590 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 411.367742411, 2023-12-14T18:36:55.590 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.408138490, 2023-12-14T18:36:55.590 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:55.591 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 220620, 2023-12-14T18:36:55.591 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 18.055429851, 2023-12-14T18:36:55.591 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000081839 2023-12-14T18:36:55.591 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.591 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:55.591 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 291915, 2023-12-14T18:36:55.591 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.049571537, 2023-12-14T18:36:55.592 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000169 2023-12-14T18:36:55.592 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.592 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1511056, 2023-12-14T18:36:55.592 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 123616 2023-12-14T18:36:55.592 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.592 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:55.593 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 23915, 2023-12-14T18:36:55.593 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 25505, 2023-12-14T18:36:55.593 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 24206196, 2023-12-14T18:36:55.593 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 227900156, 2023-12-14T18:36:55.593 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 11, 2023-12-14T18:36:55.593 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 7, 2023-12-14T18:36:55.594 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 1.821514968, 2023-12-14T18:36:55.594 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.748259151, 2023-12-14T18:36:55.594 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.628697620, 2023-12-14T18:36:55.594 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.491231752, 2023-12-14T18:36:55.594 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:55.594 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 25504, 2023-12-14T18:36:55.595 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 3.016101636, 2023-12-14T18:36:55.595 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000118259 2023-12-14T18:36:55.595 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.595 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:55.595 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 21701, 2023-12-14T18:36:55.595 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.003576868, 2023-12-14T18:36:55.595 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000164 2023-12-14T18:36:55.596 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.596 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1072, 2023-12-14T18:36:55.596 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 114912 2023-12-14T18:36:55.596 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.596 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:55.596 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 12321, 2023-12-14T18:36:55.597 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 12479, 2023-12-14T18:36:55.597 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 5167187, 2023-12-14T18:36:55.597 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 122537670, 2023-12-14T18:36:55.597 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 13, 2023-12-14T18:36:55.597 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 7, 2023-12-14T18:36:55.597 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 1.126852641, 2023-12-14T18:36:55.598 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.427968121, 2023-12-14T18:36:55.598 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.186076655, 2023-12-14T18:36:55.598 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.462986057, 2023-12-14T18:36:55.598 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:55.598 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 12475, 2023-12-14T18:36:55.598 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 2.192301226, 2023-12-14T18:36:55.599 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000175735 2023-12-14T18:36:55.599 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.599 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:55.599 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 55, 2023-12-14T18:36:55.599 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000008567, 2023-12-14T18:36:55.600 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000155 2023-12-14T18:36:55.600 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.600 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 3200, 2023-12-14T18:36:55.600 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 313696 2023-12-14T18:36:55.600 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.600 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2023-12-14T18:36:55.601 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 1, 2023-12-14T18:36:55.601 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2023-12-14T18:36:55.601 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.601 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-MDSRank": { 2023-12-14T18:36:55.601 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2023-12-14T18:36:55.601 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2023-12-14T18:36:55.602 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1587, 2023-12-14T18:36:55.602 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 98.943507177, 2023-12-14T18:36:55.602 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.062346255 2023-12-14T18:36:55.602 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:55.602 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.602 INFO:teuthology.orchestra.run.smithi160.stdout: "finisher-PurgeQueue": { 2023-12-14T18:36:55.603 INFO:teuthology.orchestra.run.smithi160.stdout: "queue_len": 0, 2023-12-14T18:36:55.603 INFO:teuthology.orchestra.run.smithi160.stdout: "complete_latency": { 2023-12-14T18:36:55.603 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 27155, 2023-12-14T18:36:55.603 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 13.166675616, 2023-12-14T18:36:55.603 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000484871 2023-12-14T18:36:55.603 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:55.603 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.604 INFO:teuthology.orchestra.run.smithi160.stdout: "mds": { 2023-12-14T18:36:55.604 INFO:teuthology.orchestra.run.smithi160.stdout: "request": 62665, 2023-12-14T18:36:55.604 INFO:teuthology.orchestra.run.smithi160.stdout: "reply": 53217, 2023-12-14T18:36:55.604 INFO:teuthology.orchestra.run.smithi160.stdout: "reply_latency": { 2023-12-14T18:36:55.604 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 53217, 2023-12-14T18:36:55.604 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 266.304089888, 2023-12-14T18:36:55.605 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.005004116 2023-12-14T18:36:55.605 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.605 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_reply": 0, 2023-12-14T18:36:55.605 INFO:teuthology.orchestra.run.smithi160.stdout: "forward": 9448, 2023-12-14T18:36:55.605 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_complete": 0, 2023-12-14T18:36:55.605 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_fetch_keys": 0, 2023-12-14T18:36:55.606 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_commit": 1117, 2023-12-14T18:36:55.606 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_split": 136, 2023-12-14T18:36:55.606 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_merge": 18, 2023-12-14T18:36:55.606 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes": 51289, 2023-12-14T18:36:55.606 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_top": 6483, 2023-12-14T18:36:55.607 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_bottom": 44794, 2023-12-14T18:36:55.607 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pin_tail": 12, 2023-12-14T18:36:55.607 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_pinned": 42027, 2023-12-14T18:36:55.607 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_expired": 17621, 2023-12-14T18:36:55.607 INFO:teuthology.orchestra.run.smithi160.stdout: "inodes_with_caps": 2, 2023-12-14T18:36:55.607 INFO:teuthology.orchestra.run.smithi160.stdout: "caps": 2, 2023-12-14T18:36:55.608 INFO:teuthology.orchestra.run.smithi160.stdout: "subtrees": 5, 2023-12-14T18:36:55.608 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse": 77418, 2023-12-14T18:36:55.608 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_hit": 67817, 2023-12-14T18:36:55.608 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_forward": 9400, 2023-12-14T18:36:55.608 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_discover": 8, 2023-12-14T18:36:55.608 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_dir_fetch": 0, 2023-12-14T18:36:55.609 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_remote_ino": 0, 2023-12-14T18:36:55.609 INFO:teuthology.orchestra.run.smithi160.stdout: "traverse_lock": 0, 2023-12-14T18:36:55.609 INFO:teuthology.orchestra.run.smithi160.stdout: "load_cent": 1000, 2023-12-14T18:36:55.609 INFO:teuthology.orchestra.run.smithi160.stdout: "q": 1, 2023-12-14T18:36:55.609 INFO:teuthology.orchestra.run.smithi160.stdout: "exported": 309, 2023-12-14T18:36:55.609 INFO:teuthology.orchestra.run.smithi160.stdout: "exported_inodes": 17203, 2023-12-14T18:36:55.610 INFO:teuthology.orchestra.run.smithi160.stdout: "imported": 280, 2023-12-14T18:36:55.610 INFO:teuthology.orchestra.run.smithi160.stdout: "imported_inodes": 0, 2023-12-14T18:36:55.610 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_dir_fetch": 0, 2023-12-14T18:36:55.610 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_backtrace_fetch": 0, 2023-12-14T18:36:55.610 INFO:teuthology.orchestra.run.smithi160.stdout: "openino_peer_discover": 0, 2023-12-14T18:36:55.610 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rfiles": 3146, 2023-12-14T18:36:55.610 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rbytes": 246, 2023-12-14T18:36:55.611 INFO:teuthology.orchestra.run.smithi160.stdout: "root_rsnaps": 6, 2023-12-14T18:36:55.611 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_fetch": 0, 2023-12-14T18:36:55.611 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_set_tag": 0, 2023-12-14T18:36:55.611 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_backtrace_repaired": 0, 2023-12-14T18:36:55.611 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_inotable_repaired": 0, 2023-12-14T18:36:55.612 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_inodes": 0, 2023-12-14T18:36:55.612 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dir_base_inodes": 0, 2023-12-14T18:36:55.612 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_dirfrag_rstats": 0, 2023-12-14T18:36:55.612 INFO:teuthology.orchestra.run.smithi160.stdout: "scrub_file_inodes": 0, 2023-12-14T18:36:55.612 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_inode_file_caps": 8, 2023-12-14T18:36:55.612 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_revoke": 76, 2023-12-14T18:36:55.612 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_grant": 40182, 2023-12-14T18:36:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_trunc": 0, 2023-12-14T18:36:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flushsnap_ack": 3114, 2023-12-14T18:36:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: "ceph_cap_op_flush_ack": 0, 2023-12-14T18:36:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps": 45955, 2023-12-14T18:36:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_caps_dirty": 45869, 2023-12-14T18:36:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_cap_release": 225, 2023-12-14T18:36:55.613 INFO:teuthology.orchestra.run.smithi160.stdout: "process_request_cap_release": 33707 2023-12-14T18:36:55.614 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.614 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_cache": { 2023-12-14T18:36:55.614 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update": 2023, 2023-12-14T18:36:55.614 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_update_receipt": 1003, 2023-12-14T18:36:55.614 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_try_discover": 504, 2023-12-14T18:36:55.614 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_send_discover": 507, 2023-12-14T18:36:55.614 INFO:teuthology.orchestra.run.smithi160.stdout: "dir_handle_discover": 2068, 2023-12-14T18:36:55.615 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays": 2, 2023-12-14T18:36:55.615 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_delayed": 0, 2023-12-14T18:36:55.615 INFO:teuthology.orchestra.run.smithi160.stdout: "num_strays_enqueuing": 0, 2023-12-14T18:36:55.615 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_created": 16480, 2023-12-14T18:36:55.615 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_enqueued": 16478, 2023-12-14T18:36:55.615 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_reintegrated": 0, 2023-12-14T18:36:55.616 INFO:teuthology.orchestra.run.smithi160.stdout: "strays_migrated": 0, 2023-12-14T18:36:55.616 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_processing": 0, 2023-12-14T18:36:55.616 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_enqueued": 0, 2023-12-14T18:36:55.616 INFO:teuthology.orchestra.run.smithi160.stdout: "num_recovering_prioritized": 0, 2023-12-14T18:36:55.616 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_started": 0, 2023-12-14T18:36:55.616 INFO:teuthology.orchestra.run.smithi160.stdout: "recovery_completed": 0, 2023-12-14T18:36:55.616 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_enqueue_scrub": 0, 2023-12-14T18:36:55.617 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_exportdir": 317, 2023-12-14T18:36:55.617 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_flush": 0, 2023-12-14T18:36:55.617 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragmentdir": 169, 2023-12-14T18:36:55.617 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_fragstats": 0, 2023-12-14T18:36:55.617 INFO:teuthology.orchestra.run.smithi160.stdout: "ireq_inodestats": 0 2023-12-14T18:36:55.617 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.618 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_log": { 2023-12-14T18:36:55.618 INFO:teuthology.orchestra.run.smithi160.stdout: "evadd": 90673, 2023-12-14T18:36:55.618 INFO:teuthology.orchestra.run.smithi160.stdout: "evex": 0, 2023-12-14T18:36:55.618 INFO:teuthology.orchestra.run.smithi160.stdout: "evtrm": 0, 2023-12-14T18:36:55.618 INFO:teuthology.orchestra.run.smithi160.stdout: "ev": 90673, 2023-12-14T18:36:55.618 INFO:teuthology.orchestra.run.smithi160.stdout: "evexg": 0, 2023-12-14T18:36:55.618 INFO:teuthology.orchestra.run.smithi160.stdout: "evexd": 0, 2023-12-14T18:36:55.619 INFO:teuthology.orchestra.run.smithi160.stdout: "segadd": 122, 2023-12-14T18:36:55.619 INFO:teuthology.orchestra.run.smithi160.stdout: "segex": 0, 2023-12-14T18:36:55.619 INFO:teuthology.orchestra.run.smithi160.stdout: "segtrm": 0, 2023-12-14T18:36:55.619 INFO:teuthology.orchestra.run.smithi160.stdout: "seg": 122, 2023-12-14T18:36:55.619 INFO:teuthology.orchestra.run.smithi160.stdout: "segexg": 0, 2023-12-14T18:36:55.620 INFO:teuthology.orchestra.run.smithi160.stdout: "segexd": 0, 2023-12-14T18:36:55.620 INFO:teuthology.orchestra.run.smithi160.stdout: "expos": 4194304, 2023-12-14T18:36:55.620 INFO:teuthology.orchestra.run.smithi160.stdout: "wrpos": 384752842, 2023-12-14T18:36:55.620 INFO:teuthology.orchestra.run.smithi160.stdout: "rdpos": 0, 2023-12-14T18:36:55.620 INFO:teuthology.orchestra.run.smithi160.stdout: "jlat": { 2023-12-14T18:36:55.620 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1260, 2023-12-14T18:36:55.620 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 97.134644649, 2023-12-14T18:36:55.621 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.077090987 2023-12-14T18:36:55.621 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.621 INFO:teuthology.orchestra.run.smithi160.stdout: "replayed": 0 2023-12-14T18:36:55.621 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.621 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_mem": { 2023-12-14T18:36:55.621 INFO:teuthology.orchestra.run.smithi160.stdout: "ino": 29045, 2023-12-14T18:36:55.621 INFO:teuthology.orchestra.run.smithi160.stdout: "ino+": 62274, 2023-12-14T18:36:55.622 INFO:teuthology.orchestra.run.smithi160.stdout: "ino-": 33229, 2023-12-14T18:36:55.622 INFO:teuthology.orchestra.run.smithi160.stdout: "dir": 1054, 2023-12-14T18:36:55.622 INFO:teuthology.orchestra.run.smithi160.stdout: "dir+": 3128, 2023-12-14T18:36:55.622 INFO:teuthology.orchestra.run.smithi160.stdout: "dir-": 2074, 2023-12-14T18:36:55.622 INFO:teuthology.orchestra.run.smithi160.stdout: "dn": 51289, 2023-12-14T18:36:55.622 INFO:teuthology.orchestra.run.smithi160.stdout: "dn+": 82953, 2023-12-14T18:36:55.622 INFO:teuthology.orchestra.run.smithi160.stdout: "dn-": 31664, 2023-12-14T18:36:55.623 INFO:teuthology.orchestra.run.smithi160.stdout: "cap": 2, 2023-12-14T18:36:55.623 INFO:teuthology.orchestra.run.smithi160.stdout: "cap+": 41203, 2023-12-14T18:36:55.623 INFO:teuthology.orchestra.run.smithi160.stdout: "cap-": 41201, 2023-12-14T18:36:55.623 INFO:teuthology.orchestra.run.smithi160.stdout: "rss": 283120, 2023-12-14T18:36:55.623 INFO:teuthology.orchestra.run.smithi160.stdout: "heap": 198964 2023-12-14T18:36:55.623 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.623 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_server": { 2023-12-14T18:36:55.624 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_client_request": 63898, 2023-12-14T18:36:55.624 INFO:teuthology.orchestra.run.smithi160.stdout: "dispatch_server_request": 46, 2023-12-14T18:36:55.624 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_request": 62665, 2023-12-14T18:36:55.624 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_client_session": 313, 2023-12-14T18:36:55.624 INFO:teuthology.orchestra.run.smithi160.stdout: "handle_peer_request": 126, 2023-12-14T18:36:55.624 INFO:teuthology.orchestra.run.smithi160.stdout: "req_create_latency": { 2023-12-14T18:36:55.624 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 20597, 2023-12-14T18:36:55.624 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 77.443809531, 2023-12-14T18:36:55.625 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.003759955 2023-12-14T18:36:55.625 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.625 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getattr_latency": { 2023-12-14T18:36:55.625 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 5, 2023-12-14T18:36:55.625 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.039988754, 2023-12-14T18:36:55.625 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.007997750 2023-12-14T18:36:55.625 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.626 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getfilelock_latency": { 2023-12-14T18:36:55.626 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.626 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.626 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.626 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.626 INFO:teuthology.orchestra.run.smithi160.stdout: "req_link_latency": { 2023-12-14T18:36:55.626 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.627 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.627 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.627 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.627 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookup_latency": { 2023-12-14T18:36:55.627 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 14095, 2023-12-14T18:36:55.627 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 74.440330171, 2023-12-14T18:36:55.627 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.005281328 2023-12-14T18:36:55.628 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.628 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookuphash_latency": { 2023-12-14T18:36:55.628 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.628 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.628 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.628 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.628 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupino_latency": { 2023-12-14T18:36:55.629 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 1, 2023-12-14T18:36:55.629 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000097387, 2023-12-14T18:36:55.629 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000097387 2023-12-14T18:36:55.629 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.629 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupname_latency": { 2023-12-14T18:36:55.629 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.629 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.630 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.630 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.630 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupparent_latency": { 2023-12-14T18:36:55.630 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.630 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.630 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.630 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.631 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lookupsnap_latency": { 2023-12-14T18:36:55.631 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.631 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.631 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.631 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.631 INFO:teuthology.orchestra.run.smithi160.stdout: "req_lssnap_latency": { 2023-12-14T18:36:55.631 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.632 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.632 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.632 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.632 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mkdir_latency": { 2023-12-14T18:36:55.632 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.632 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.632 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.632 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.633 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mknod_latency": { 2023-12-14T18:36:55.633 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.633 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.633 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.633 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.633 INFO:teuthology.orchestra.run.smithi160.stdout: "req_mksnap_latency": { 2023-12-14T18:36:55.633 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.634 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.634 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.634 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.634 INFO:teuthology.orchestra.run.smithi160.stdout: "req_open_latency": { 2023-12-14T18:36:55.634 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.634 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.634 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.635 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.635 INFO:teuthology.orchestra.run.smithi160.stdout: "req_readdir_latency": { 2023-12-14T18:36:55.635 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2039, 2023-12-14T18:36:55.635 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 16.972817411, 2023-12-14T18:36:55.635 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.008324088 2023-12-14T18:36:55.635 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.635 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rename_latency": { 2023-12-14T18:36:55.636 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.636 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.636 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.636 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.636 INFO:teuthology.orchestra.run.smithi160.stdout: "req_renamesnap_latency": { 2023-12-14T18:36:55.636 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.636 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.636 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.637 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.637 INFO:teuthology.orchestra.run.smithi160.stdout: "req_snapdiff_latency": { 2023-12-14T18:36:55.637 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.637 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.637 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.637 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.637 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmdir_latency": { 2023-12-14T18:36:55.638 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 2, 2023-12-14T18:36:55.638 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.132350539, 2023-12-14T18:36:55.638 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.066175269 2023-12-14T18:36:55.638 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.638 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmsnap_latency": { 2023-12-14T18:36:55.638 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.638 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.639 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.639 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.639 INFO:teuthology.orchestra.run.smithi160.stdout: "req_rmxattr_latency": { 2023-12-14T18:36:55.639 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.639 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.639 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.639 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.639 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setattr_latency": { 2023-12-14T18:36:55.640 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.640 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.640 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.640 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.640 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setdirlayout_latency": { 2023-12-14T18:36:55.640 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.640 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.641 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.641 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.641 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setfilelock_latency": { 2023-12-14T18:36:55.641 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.641 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.641 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.641 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.642 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setlayout_latency": { 2023-12-14T18:36:55.642 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.642 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.642 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.642 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.642 INFO:teuthology.orchestra.run.smithi160.stdout: "req_setxattr_latency": { 2023-12-14T18:36:55.642 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.642 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.643 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.643 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.643 INFO:teuthology.orchestra.run.smithi160.stdout: "req_symlink_latency": { 2023-12-14T18:36:55.643 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.643 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.643 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.643 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.643 INFO:teuthology.orchestra.run.smithi160.stdout: "req_unlink_latency": { 2023-12-14T18:36:55.644 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 16478, 2023-12-14T18:36:55.644 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 97.274696095, 2023-12-14T18:36:55.644 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.005903307 2023-12-14T18:36:55.644 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.644 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_revoke_eviction": 0, 2023-12-14T18:36:55.644 INFO:teuthology.orchestra.run.smithi160.stdout: "cap_acquisition_throttle": 0, 2023-12-14T18:36:55.644 INFO:teuthology.orchestra.run.smithi160.stdout: "req_getvxattr_latency": { 2023-12-14T18:36:55.645 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.645 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.645 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.645 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:55.645 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.645 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_sessions": { 2023-12-14T18:36:55.645 INFO:teuthology.orchestra.run.smithi160.stdout: "session_count": 2, 2023-12-14T18:36:55.645 INFO:teuthology.orchestra.run.smithi160.stdout: "session_add": 4, 2023-12-14T18:36:55.646 INFO:teuthology.orchestra.run.smithi160.stdout: "session_remove": 2, 2023-12-14T18:36:55.646 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_open": 2, 2023-12-14T18:36:55.646 INFO:teuthology.orchestra.run.smithi160.stdout: "sessions_stale": 0, 2023-12-14T18:36:55.646 INFO:teuthology.orchestra.run.smithi160.stdout: "total_load": 13867, 2023-12-14T18:36:55.646 INFO:teuthology.orchestra.run.smithi160.stdout: "average_load": 6933, 2023-12-14T18:36:55.646 INFO:teuthology.orchestra.run.smithi160.stdout: "avg_session_uptime": 605 2023-12-14T18:36:55.646 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.647 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2023-12-14T18:36:55.647 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:55.647 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:55.647 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:55.647 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:55.647 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:55.647 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:55.648 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:55.648 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:55.648 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:55.648 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:55.648 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:55.648 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:55.648 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:55.649 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:55.649 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:55.649 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:55.649 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:55.649 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:55.649 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:55.649 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:55.649 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:55.650 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:55.650 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:55.650 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:55.650 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:55.650 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:55.650 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:55.650 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:55.650 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:55.651 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:55.651 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:55.651 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2023-12-14T18:36:55.651 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:55.651 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:55.651 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:55.651 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:55.651 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 1032453, 2023-12-14T18:36:55.652 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 21493, 2023-12-14T18:36:55.652 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 440, 2023-12-14T18:36:55.652 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 5, 2023-12-14T18:36:55.652 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2023-12-14T18:36:55.652 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2023-12-14T18:36:55.652 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:55.652 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:55.652 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:55.653 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:55.653 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 9200, 2023-12-14T18:36:55.653 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 175, 2023-12-14T18:36:55.653 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:55.653 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:55.653 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:55.653 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2023-12-14T18:36:55.654 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 102794583, 2023-12-14T18:36:55.654 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 908041, 2023-12-14T18:36:55.654 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:55.654 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2023-12-14T18:36:55.654 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:55.654 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2023-12-14T18:36:55.654 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.654 INFO:teuthology.orchestra.run.smithi160.stdout: "objecter": { 2023-12-14T18:36:55.655 INFO:teuthology.orchestra.run.smithi160.stdout: "op_active": 0, 2023-12-14T18:36:55.655 INFO:teuthology.orchestra.run.smithi160.stdout: "op_laggy": 0, 2023-12-14T18:36:55.655 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send": 36295, 2023-12-14T18:36:55.655 INFO:teuthology.orchestra.run.smithi160.stdout: "op_send_bytes": 398027025, 2023-12-14T18:36:55.655 INFO:teuthology.orchestra.run.smithi160.stdout: "op_resend": 27, 2023-12-14T18:36:55.655 INFO:teuthology.orchestra.run.smithi160.stdout: "op_reply": 36295, 2023-12-14T18:36:55.655 INFO:teuthology.orchestra.run.smithi160.stdout: "op_latency": { 2023-12-14T18:36:55.655 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 36295, 2023-12-14T18:36:55.656 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 276.713718653, 2023-12-14T18:36:55.656 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.007624017 2023-12-14T18:36:55.656 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.656 INFO:teuthology.orchestra.run.smithi160.stdout: "op_inflight": 0, 2023-12-14T18:36:55.656 INFO:teuthology.orchestra.run.smithi160.stdout: "oplen_avg": { 2023-12-14T18:36:55.656 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 36295, 2023-12-14T18:36:55.656 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 37511 2023-12-14T18:36:55.656 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.656 INFO:teuthology.orchestra.run.smithi160.stdout: "op": 36295, 2023-12-14T18:36:55.657 INFO:teuthology.orchestra.run.smithi160.stdout: "op_r": 232, 2023-12-14T18:36:55.657 INFO:teuthology.orchestra.run.smithi160.stdout: "op_w": 36063, 2023-12-14T18:36:55.657 INFO:teuthology.orchestra.run.smithi160.stdout: "op_rmw": 0, 2023-12-14T18:36:55.657 INFO:teuthology.orchestra.run.smithi160.stdout: "op_pg": 0, 2023-12-14T18:36:55.657 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_stat": 0, 2023-12-14T18:36:55.657 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_create": 10, 2023-12-14T18:36:55.657 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_read": 232, 2023-12-14T18:36:55.658 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_write": 1493, 2023-12-14T18:36:55.658 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writefull": 112, 2023-12-14T18:36:55.658 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_writesame": 0, 2023-12-14T18:36:55.658 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_append": 0, 2023-12-14T18:36:55.658 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_zero": 0, 2023-12-14T18:36:55.658 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_truncate": 8, 2023-12-14T18:36:55.658 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_delete": 33270, 2023-12-14T18:36:55.659 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_mapext": 0, 2023-12-14T18:36:55.659 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_sparse_read": 0, 2023-12-14T18:36:55.659 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_clonerange": 0, 2023-12-14T18:36:55.659 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_getxattr": 0, 2023-12-14T18:36:55.659 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_setxattr": 20, 2023-12-14T18:36:55.659 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_cmpxattr": 0, 2023-12-14T18:36:55.659 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_rmxattr": 0, 2023-12-14T18:36:55.659 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_resetxattrs": 0, 2023-12-14T18:36:55.660 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_call": 0, 2023-12-14T18:36:55.660 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_watch": 0, 2023-12-14T18:36:55.660 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_notify": 0, 2023-12-14T18:36:55.660 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_src_cmpxattr": 0, 2023-12-14T18:36:55.660 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls": 0, 2023-12-14T18:36:55.660 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_pgls_filter": 0, 2023-12-14T18:36:55.660 INFO:teuthology.orchestra.run.smithi160.stdout: "osdop_other": 0, 2023-12-14T18:36:55.660 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_active": 0, 2023-12-14T18:36:55.661 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_send": 0, 2023-12-14T18:36:55.661 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_resend": 0, 2023-12-14T18:36:55.661 INFO:teuthology.orchestra.run.smithi160.stdout: "linger_ping": 0, 2023-12-14T18:36:55.661 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_active": 0, 2023-12-14T18:36:55.661 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_send": 0, 2023-12-14T18:36:55.661 INFO:teuthology.orchestra.run.smithi160.stdout: "poolop_resend": 0, 2023-12-14T18:36:55.661 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_active": 0, 2023-12-14T18:36:55.661 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_send": 0, 2023-12-14T18:36:55.662 INFO:teuthology.orchestra.run.smithi160.stdout: "poolstat_resend": 0, 2023-12-14T18:36:55.662 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_active": 0, 2023-12-14T18:36:55.662 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_send": 0, 2023-12-14T18:36:55.662 INFO:teuthology.orchestra.run.smithi160.stdout: "statfs_resend": 0, 2023-12-14T18:36:55.662 INFO:teuthology.orchestra.run.smithi160.stdout: "command_active": 0, 2023-12-14T18:36:55.662 INFO:teuthology.orchestra.run.smithi160.stdout: "command_send": 0, 2023-12-14T18:36:55.662 INFO:teuthology.orchestra.run.smithi160.stdout: "command_resend": 0, 2023-12-14T18:36:55.662 INFO:teuthology.orchestra.run.smithi160.stdout: "map_epoch": 96, 2023-12-14T18:36:55.662 INFO:teuthology.orchestra.run.smithi160.stdout: "map_full": 0, 2023-12-14T18:36:55.663 INFO:teuthology.orchestra.run.smithi160.stdout: "map_inc": 12, 2023-12-14T18:36:55.663 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_sessions": 12, 2023-12-14T18:36:55.663 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_open": 12, 2023-12-14T18:36:55.663 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_session_close": 0, 2023-12-14T18:36:55.663 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_laggy": 0, 2023-12-14T18:36:55.663 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_wr": 2317, 2023-12-14T18:36:55.663 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_rd": 0, 2023-12-14T18:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_del": 49 2023-12-14T18:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout: "oft": { 2023-12-14T18:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_objs": 1, 2023-12-14T18:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_kv_pairs": 0, 2023-12-14T18:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_updates": 0, 2023-12-14T18:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout: "omap_total_removes": 0 2023-12-14T18:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.664 INFO:teuthology.orchestra.run.smithi160.stdout: "osd-slow-ops": { 2023-12-14T18:36:55.665 INFO:teuthology.orchestra.run.smithi160.stdout: "slow_ops_count": 63197 2023-12-14T18:36:55.665 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.665 INFO:teuthology.orchestra.run.smithi160.stdout: "purge_queue": { 2023-12-14T18:36:55.665 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops": 0, 2023-12-14T18:36:55.665 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_ops_high_water": 16, 2023-12-14T18:36:55.665 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing": 0, 2023-12-14T18:36:55.665 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executing_high_water": 8, 2023-12-14T18:36:55.666 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_executed": 16478, 2023-12-14T18:36:55.666 INFO:teuthology.orchestra.run.smithi160.stdout: "pq_item_in_journal": 0 2023-12-14T18:36:55.666 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.666 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:55.666 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:55.666 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2023-12-14T18:36:55.666 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:55.666 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 161683, 2023-12-14T18:36:55.666 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 117519775, 2023-12-14T18:36:55.667 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:55.667 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 161683, 2023-12-14T18:36:55.667 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:55.667 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:55.667 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 161683, 2023-12-14T18:36:55.667 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 117519775, 2023-12-14T18:36:55.667 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:55.667 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.668 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.668 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.668 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:55.668 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.668 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_bytes": { 2023-12-14T18:36:55.668 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:55.669 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2023-12-14T18:36:55.669 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:55.669 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:55.669 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:55.669 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:55.669 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:55.669 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 36295, 2023-12-14T18:36:55.669 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 400055647, 2023-12-14T18:36:55.670 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 36295, 2023-12-14T18:36:55.670 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 400055647, 2023-12-14T18:36:55.670 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:55.670 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.670 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.670 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.670 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:55.670 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.671 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-objecter_ops": { 2023-12-14T18:36:55.671 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:55.671 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 1024, 2023-12-14T18:36:55.671 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:55.671 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 0, 2023-12-14T18:36:55.671 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 0, 2023-12-14T18:36:55.671 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:55.671 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 0, 2023-12-14T18:36:55.671 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 36295, 2023-12-14T18:36:55.672 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 36295, 2023-12-14T18:36:55.672 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 36295, 2023-12-14T18:36:55.672 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 36295, 2023-12-14T18:36:55.672 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:55.672 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.672 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.672 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.672 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:55.673 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.673 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle": { 2023-12-14T18:36:55.673 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:55.673 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2023-12-14T18:36:55.673 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:55.673 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 16478, 2023-12-14T18:36:55.673 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 2028622, 2023-12-14T18:36:55.673 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:55.673 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 16478, 2023-12-14T18:36:55.674 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:55.674 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:55.674 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 233, 2023-12-14T18:36:55.674 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 2028622, 2023-12-14T18:36:55.674 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:55.674 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.674 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.674 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.675 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:55.675 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:55.675 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-write_buf_throttle-0x5561554ae0a0": { 2023-12-14T18:36:55.675 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:55.675 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 3758096384, 2023-12-14T18:36:55.675 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:55.675 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 90673, 2023-12-14T18:36:55.675 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 380558538, 2023-12-14T18:36:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 90673, 2023-12-14T18:36:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 1260, 2023-12-14T18:36:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 380558538, 2023-12-14T18:36:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:55.676 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:55.677 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:55.677 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:55.677 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:55.677 INFO:teuthology.orchestra.run.smithi160.stdout:} 2023-12-14T18:36:55.677 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:55 smithi097 ceph-mon[43486]: pgmap v325: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:55.949 INFO:tasks.check_counter:Daemon mds.i mds.exported=309 2023-12-14T18:36:55.950 INFO:tasks.check_counter:Daemon mds.i mds.imported=280 2023-12-14T18:36:55.950 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=2023 2023-12-14T18:36:55.950 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=1003 2023-12-14T18:36:55.950 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2023-12-14T18:36:55.950 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2023-12-14T18:36:55.950 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2023-12-14T18:36:55.950 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=136 2023-12-14T18:36:55.950 DEBUG:tasks.check_counter:Getting stats from l 2023-12-14T18:36:55.951 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1faa353368ad4f5aeb56d7107b638336f58659ed shell --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 -- ceph daemon mds.l perf dump 2023-12-14T18:36:56.261 INFO:teuthology.orchestra.run.smithi160.stderr:Inferring config /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/mon.c/config 2023-12-14T18:36:57.247 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:56 smithi160 ceph-mon[43840]: pgmap v326: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:57.260 INFO:teuthology.orchestra.run.smithi160.stdout:{ 2023-12-14T18:36:57.261 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-0": { 2023-12-14T18:36:57.261 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 4, 2023-12-14T18:36:57.261 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 4, 2023-12-14T18:36:57.261 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 2992, 2023-12-14T18:36:57.261 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 448, 2023-12-14T18:36:57.261 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 4, 2023-12-14T18:36:57.261 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 0, 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.001404232, 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.000040977, 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.000555347, 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 3, 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000778464, 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000259488 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.262 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 2992, 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 448 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-1": { 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 3, 2023-12-14T18:36:57.263 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 158, 2023-12-14T18:36:57.264 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 480, 2023-12-14T18:36:57.264 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 255664, 2023-12-14T18:36:57.264 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 6, 2023-12-14T18:36:57.264 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 3, 2023-12-14T18:36:57.264 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.011508967, 2023-12-14T18:36:57.264 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.007184562, 2023-12-14T18:36:57.264 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.001913620, 2023-12-14T18:36:57.264 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2023-12-14T18:36:57.264 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 155, 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.029170374, 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000188195 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.265 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 480, 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 255664 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: "AsyncMessenger::Worker-2": { 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_messages": 216, 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages": 206, 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_bytes": 1420080, 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_bytes": 117376, 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_created_connections": 2, 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_active_connections": 1, 2023-12-14T18:36:57.266 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_total_time": 0.039965184, 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_send_time": 0.011066135, 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_recv_time": 0.032466150, 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_running_fast_dispatch_time": 0.005103451, 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_messages_queue_lat": { 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 205, 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.012649123, 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000061703 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_handle_ack_lat": { 2023-12-14T18:36:57.267 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_recv_encrypted_bytes": 1420080, 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: "msgr_send_encrypted_bytes": 117376 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: "cct": { 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: "total_workers": 0, 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: "unhealthy_workers": 0 2023-12-14T18:36:57.268 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.269 INFO:teuthology.orchestra.run.smithi160.stdout: "mempool": { 2023-12-14T18:36:57.269 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_bytes": 0, 2023-12-14T18:36:57.269 INFO:teuthology.orchestra.run.smithi160.stdout: "bloom_filter_items": 0, 2023-12-14T18:36:57.269 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_bytes": 0, 2023-12-14T18:36:57.269 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_alloc_items": 0, 2023-12-14T18:36:57.269 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_bytes": 0, 2023-12-14T18:36:57.269 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_data_items": 0, 2023-12-14T18:36:57.269 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_bytes": 0, 2023-12-14T18:36:57.269 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_onode_items": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_bytes": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_meta_items": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_bytes": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_other_items": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_bytes": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_cache_buffer_items": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_bytes": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_extent_items": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_bytes": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_blob_items": 0, 2023-12-14T18:36:57.270 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_bytes": 0, 2023-12-14T18:36:57.271 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_shared_blob_items": 0, 2023-12-14T18:36:57.271 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_bytes": 0, 2023-12-14T18:36:57.271 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_inline_bl_items": 0, 2023-12-14T18:36:57.271 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_bytes": 0, 2023-12-14T18:36:57.271 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_fsck_items": 0, 2023-12-14T18:36:57.271 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_bytes": 0, 2023-12-14T18:36:57.271 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_txc_items": 0, 2023-12-14T18:36:57.271 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_bytes": 0, 2023-12-14T18:36:57.271 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_deferred_items": 0, 2023-12-14T18:36:57.272 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_bytes": 0, 2023-12-14T18:36:57.272 INFO:teuthology.orchestra.run.smithi160.stdout: "bluestore_writing_items": 0, 2023-12-14T18:36:57.272 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_bytes": 0, 2023-12-14T18:36:57.272 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_items": 0, 2023-12-14T18:36:57.272 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_bytes": 0, 2023-12-14T18:36:57.272 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_reader_items": 0, 2023-12-14T18:36:57.272 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_bytes": 0, 2023-12-14T18:36:57.272 INFO:teuthology.orchestra.run.smithi160.stdout: "bluefs_file_writer_items": 0, 2023-12-14T18:36:57.272 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_bytes": 149007, 2023-12-14T18:36:57.273 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_anon_items": 30, 2023-12-14T18:36:57.273 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_bytes": 0, 2023-12-14T18:36:57.273 INFO:teuthology.orchestra.run.smithi160.stdout: "buffer_meta_items": 0, 2023-12-14T18:36:57.273 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_bytes": 0, 2023-12-14T18:36:57.273 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_items": 0, 2023-12-14T18:36:57.273 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_bytes": 0, 2023-12-14T18:36:57.273 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_mapbl_items": 0, 2023-12-14T18:36:57.273 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_bytes": 0, 2023-12-14T18:36:57.273 INFO:teuthology.orchestra.run.smithi160.stdout: "osd_pglog_items": 0, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_bytes": 0, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_items": 0, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_bytes": 0, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "osdmap_mapping_items": 0, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_bytes": 0, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "pgmap_items": 0, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_bytes": 928, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "mds_co_items": 2, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_bytes": 0, 2023-12-14T18:36:57.274 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_1_items": 0, 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_bytes": 0, 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "unittest_2_items": 0 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: }, 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "throttle-msgr_dispatch_throttler-mds": { 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "val": 0, 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "max": 104857600, 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "get_started": 0, 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "get": 219, 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "get_sum": 1389624, 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_fail": 0, 2023-12-14T18:36:57.275 INFO:teuthology.orchestra.run.smithi160.stdout: "get_or_fail_success": 219, 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: "take": 0, 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: "take_sum": 0, 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: "put": 219, 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: "put_sum": 1389624, 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: "wait": { 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: "avgcount": 0, 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: "sum": 0.000000000, 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: "avgtime": 0.000000000 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:57.276 INFO:teuthology.orchestra.run.smithi160.stdout: } 2023-12-14T18:36:57.277 INFO:teuthology.orchestra.run.smithi160.stdout:} 2023-12-14T18:36:57.314 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:56 smithi093 ceph-mon[42070]: pgmap v326: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:57.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:56 smithi097 ceph-mon[43486]: pgmap v326: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:36:57.641 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.l 2023-12-14T18:36:57.641 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.l 2023-12-14T18:36:57.641 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.l 2023-12-14T18:36:57.641 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.l 2023-12-14T18:36:57.642 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2023-12-14T18:36:57.642 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2023-12-14T18:36:57.642 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2023-12-14T18:36:57.642 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.l 2023-12-14T18:36:57.642 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-14T18:36:57.652 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-14T18:36:57.652 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:36:57.652 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:36:57.757 DEBUG:teuthology.orchestra.run.smithi093:> set -ex 2023-12-14T18:36:57.757 DEBUG:teuthology.orchestra.run.smithi093:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-14T18:36:57.818 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:36:58.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.002+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2806344023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ffdc0 msgr2=0x7f20a4103d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.002+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2806344023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ffdc0 0x7f20a4103d30 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f20940099c0 tx=0x7f2094009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.002+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2806344023 shutdown_connections 2023-12-14T18:36:58.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.002+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2806344023 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a4104410 0x7f20a410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.004 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.002+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2806344023 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20a40ff420 0x7f20a40ff7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.002+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2806344023 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ffdc0 0x7f20a4103d30 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.002+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2806344023 >> 172.21.15.93:0/2806344023 conn(0x7f20a40761f0 msgr2=0x7f20a40765f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:58.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.002+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2806344023 shutdown_connections 2023-12-14T18:36:58.005 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.002+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2806344023 wait complete. 2023-12-14T18:36:58.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.003+0000 7f20a8e8b700 1 Processor -- start 2023-12-14T18:36:58.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.003+0000 7f20a8e8b700 1 -- start start 2023-12-14T18:36:58.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a8e8b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ff420 0x7f20a4195300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a8e8b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20a40ffdc0 0x7f20a4195840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.007 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a8e8b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a4104410 0x7f20a4199ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a8e8b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a419a2e0 con 0x7f20a40ff420 2023-12-14T18:36:58.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a8e8b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a419a450 con 0x7f20a40ffdc0 2023-12-14T18:36:58.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a8e8b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20a419a610 con 0x7f20a4104410 2023-12-14T18:36:58.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ff420 0x7f20a4195300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a259c700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ff420 0x7f20a4195300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50366/0 (socket says 172.21.15.93:50366) 2023-12-14T18:36:58.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a259c700 1 -- 172.21.15.93:0/2529327922 learned_addr learned my addr 172.21.15.93:0/2529327922 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:58.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a1d9b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20a40ffdc0 0x7f20a4195840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a2d9d700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a4104410 0x7f20a4199ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.008 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a259c700 1 -- 172.21.15.93:0/2529327922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a4104410 msgr2=0x7f20a4199ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a259c700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a4104410 0x7f20a4199ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a259c700 1 -- 172.21.15.93:0/2529327922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20a40ffdc0 msgr2=0x7f20a4195840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a259c700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20a40ffdc0 0x7f20a4195840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a259c700 1 -- 172.21.15.93:0/2529327922 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20940095f0 con 0x7f20a40ff420 2023-12-14T18:36:58.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a1d9b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20a40ffdc0 0x7f20a4195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:58.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a259c700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ff420 0x7f20a4195300 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f209800d950 tx=0x7f209800dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:58.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20937fe700 1 -- 172.21.15.93:0/2529327922 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2098009b40 con 0x7f20a40ff420 2023-12-14T18:36:58.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20937fe700 1 -- 172.21.15.93:0/2529327922 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f2098009ca0 con 0x7f20a40ff420 2023-12-14T18:36:58.009 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20937fe700 1 -- 172.21.15.93:0/2529327922 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f209800f620 con 0x7f20a40ff420 2023-12-14T18:36:58.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20a410f190 con 0x7f20a40ff420 2023-12-14T18:36:58.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.004+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20a410f000 con 0x7f20a40ff420 2023-12-14T18:36:58.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.005+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20a410f320 con 0x7f20a40ff420 2023-12-14T18:36:58.010 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.007+0000 7f20937fe700 1 -- 172.21.15.93:0/2529327922 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f2098010460 con 0x7f20a40ff420 2023-12-14T18:36:58.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.008+0000 7f20937fe700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f208c0a1340 0x7f208c0a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.008+0000 7f20a1d9b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f208c0a1340 0x7f208c0a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.008+0000 7f20937fe700 1 -- 172.21.15.93:0/2529327922 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7f20980cd3d0 con 0x7f20a40ff420 2023-12-14T18:36:58.011 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.008+0000 7f20a1d9b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f208c0a1340 0x7f208c0a37f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2094009990 tx=0x7f2094005fb0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:58.013 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.011+0000 7f20937fe700 1 -- 172.21.15.93:0/2529327922 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f2098096460 con 0x7f20a40ff420 2023-12-14T18:36:58.205 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.203+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20a4067e40 con 0x7f20a40ff420 2023-12-14T18:36:58.207 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.205+0000 7f20937fe700 1 -- 172.21.15.93:0/2529327922 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24846 (secure 0 0 0) 0x7f2098016090 con 0x7f20a40ff420 2023-12-14T18:36:58.207 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:36:58.207 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":96,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:36:40.942497+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1584909638":"2023-12-15T18:26:27.742320+0000","172.21.15.93:0/1420703523":"2023-12-15T18:26:27.742320+0000","172.21.15.93:0/932543927":"2023-12-15T18:26:27.742320+0000","172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6801/3802499407":"2023-12-15T18:26:27.742320+0000","172.21.15.93:6800/3802499407":"2023-12-15T18:26:27.742320+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/978480379":"2023-12-15T18:26:27.742320+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:36:58.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f208c0a1340 msgr2=0x7f208c0a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f208c0a1340 0x7f208c0a37f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f2094009990 tx=0x7f2094005fb0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ff420 msgr2=0x7f20a4195300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ff420 0x7f20a4195300 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f209800d950 tx=0x7f209800dd10 comp rx=0 tx=0).stop 2023-12-14T18:36:58.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 shutdown_connections 2023-12-14T18:36:58.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f208c0a1340 0x7f208c0a37f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f20a40ff420 0x7f20a4195300 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f20a4104410 0x7f20a4199ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 --2- 172.21.15.93:0/2529327922 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f20a40ffdc0 0x7f20a4195840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 >> 172.21.15.93:0/2529327922 conn(0x7f20a40761f0 msgr2=0x7f20a40fc9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:58.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 shutdown_connections 2023-12-14T18:36:58.210 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.207+0000 7f20a8e8b700 1 -- 172.21.15.93:0/2529327922 wait complete. 2023-12-14T18:36:58.222 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-14T18:21:24.200509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-14T18:24:34.407045+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-14T18:24:35.728548+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2023-12-14T18:24:38.796189+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '94', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 94, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-12-14T18:36:58.222 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-14T18:36:58.347 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.345+0000 7f1be9759700 1 -- 172.21.15.93:0/4195555559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4100e70 msgr2=0x7f1be4104e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.345+0000 7f1be9759700 1 --2- 172.21.15.93:0/4195555559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4100e70 0x7f1be4104e30 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f1bcc0099c0 tx=0x7f1bcc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.345+0000 7f1be9759700 1 -- 172.21.15.93:0/4195555559 shutdown_connections 2023-12-14T18:36:58.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.345+0000 7f1be9759700 1 --2- 172.21.15.93:0/4195555559 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be4105510 0x7f1be410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.345+0000 7f1be9759700 1 --2- 172.21.15.93:0/4195555559 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be41004d0 0x7f1be41008a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.345+0000 7f1be9759700 1 --2- 172.21.15.93:0/4195555559 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4100e70 0x7f1be4104e30 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.345+0000 7f1be9759700 1 -- 172.21.15.93:0/4195555559 >> 172.21.15.93:0/4195555559 conn(0x7f1be40fbd20 msgr2=0x7f1be40fe130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:58.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.345+0000 7f1be9759700 1 -- 172.21.15.93:0/4195555559 shutdown_connections 2023-12-14T18:36:58.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.345+0000 7f1be9759700 1 -- 172.21.15.93:0/4195555559 wait complete. 2023-12-14T18:36:58.348 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be9759700 1 Processor -- start 2023-12-14T18:36:58.350 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be9759700 1 -- start start 2023-12-14T18:36:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be9759700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be41004d0 0x7f1be4196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be9759700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be4100e70 0x7f1be4196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be9759700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4105510 0x7f1be419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be9759700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419b310 con 0x7f1be4105510 2023-12-14T18:36:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be9759700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419b480 con 0x7f1be4100e70 2023-12-14T18:36:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be9759700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1be419b640 con 0x7f1be41004d0 2023-12-14T18:36:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4105510 0x7f1be419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.351 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4105510 0x7f1be419acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50370/0 (socket says 172.21.15.93:50370) 2023-12-14T18:36:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be37fe700 1 -- 172.21.15.93:0/1885692237 learned_addr learned my addr 172.21.15.93:0/1885692237 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be2ffd700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be41004d0 0x7f1be4196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.346+0000 7f1be27fc700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be4100e70 0x7f1be4196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1be37fe700 1 -- 172.21.15.93:0/1885692237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be41004d0 msgr2=0x7f1be4196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1be37fe700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be41004d0 0x7f1be4196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1be37fe700 1 -- 172.21.15.93:0/1885692237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be4100e70 msgr2=0x7f1be4196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1be37fe700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be4100e70 0x7f1be4196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1be37fe700 1 -- 172.21.15.93:0/1885692237 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1bcc0095f0 con 0x7f1be4105510 2023-12-14T18:36:58.352 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1be37fe700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4105510 0x7f1be419acd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400ef30 tx=0x7f1bd400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:58.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1bdbfff700 1 -- 172.21.15.93:0/1885692237 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4018070 con 0x7f1be4105510 2023-12-14T18:36:58.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1be4110290 con 0x7f1be4105510 2023-12-14T18:36:58.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1bdbfff700 1 -- 172.21.15.93:0/1885692237 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1bd400f040 con 0x7f1be4105510 2023-12-14T18:36:58.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1bdbfff700 1 -- 172.21.15.93:0/1885692237 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400cd00 con 0x7f1be4105510 2023-12-14T18:36:58.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.347+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1be4110100 con 0x7f1be4105510 2023-12-14T18:36:58.353 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.348+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1be4110420 con 0x7f1be4105510 2023-12-14T18:36:58.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.350+0000 7f1bdbfff700 1 -- 172.21.15.93:0/1885692237 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1bd4003a00 con 0x7f1be4105510 2023-12-14T18:36:58.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.351+0000 7f1bdbfff700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1bd00a12c0 0x7f1bd00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.354 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.351+0000 7f1be2ffd700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1bd00a12c0 0x7f1bd00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.351+0000 7f1bdbfff700 1 -- 172.21.15.93:0/1885692237 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7f1bd40cd510 con 0x7f1be4105510 2023-12-14T18:36:58.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.351+0000 7f1be2ffd700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1bd00a12c0 0x7f1bd00a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00ac50 tx=0x7f1bdc00a380 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:58.355 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.354+0000 7f1bdbfff700 1 -- 172.21.15.93:0/1885692237 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1bd4096630 con 0x7f1be4105510 2023-12-14T18:36:58.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.542+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1be4067e40 con 0x7f1be4105510 2023-12-14T18:36:58.544 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.542+0000 7f1bdbfff700 1 -- 172.21.15.93:0/1885692237 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v96) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1bd4099c50 con 0x7f1be4105510 2023-12-14T18:36:58.544 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 1 2023-12-14T18:36:58.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1bd00a12c0 msgr2=0x7f1bd00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.546 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1bd00a12c0 0x7f1bd00a3770 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1bdc00ac50 tx=0x7f1bdc00a380 comp rx=0 tx=0).stop 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4105510 msgr2=0x7f1be419acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4105510 0x7f1be419acd0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400ef30 tx=0x7f1bd400c3a0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 shutdown_connections 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1bd00a12c0 0x7f1bd00a3770 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1be4105510 0x7f1be419acd0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1be41004d0 0x7f1be4196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 --2- 172.21.15.93:0/1885692237 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1be4100e70 0x7f1be4196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 >> 172.21.15.93:0/1885692237 conn(0x7f1be40fbd20 msgr2=0x7f1be40fc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 shutdown_connections 2023-12-14T18:36:58.547 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.544+0000 7f1be9759700 1 -- 172.21.15.93:0/1885692237 wait complete. 2023-12-14T18:36:58.558 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-14T18:36:58.684 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.682+0000 7f73e5e1e700 1 -- 172.21.15.93:0/2130050396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ff430 msgr2=0x7f73e00ff800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.682+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/2130050396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ff430 0x7f73e00ff800 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f73d4009990 tx=0x7f73d4009ca0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.682+0000 7f73e5e1e700 1 -- 172.21.15.93:0/2130050396 shutdown_connections 2023-12-14T18:36:58.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.682+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/2130050396 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ff430 0x7f73e00ff800 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.682+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/2130050396 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73e010acf0 0x7f73e010d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.682+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/2130050396 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e00ffd40 0x7f73e010a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.682+0000 7f73e5e1e700 1 -- 172.21.15.93:0/2130050396 >> 172.21.15.93:0/2130050396 conn(0x7f73e00749c0 msgr2=0x7f73e0074dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:58.685 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.682+0000 7f73e5e1e700 1 -- 172.21.15.93:0/2130050396 shutdown_connections 2023-12-14T18:36:58.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.682+0000 7f73e5e1e700 1 -- 172.21.15.93:0/2130050396 wait complete. 2023-12-14T18:36:58.686 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73e5e1e700 1 Processor -- start 2023-12-14T18:36:58.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73e5e1e700 1 -- start start 2023-12-14T18:36:58.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73e5e1e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73e00ff430 0x7f73e0196340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73e5e1e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ffd40 0x7f73e0196880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73e5e1e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e010acf0 0x7f73e019ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73e5e1e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e019b320 con 0x7f73e00ffd40 2023-12-14T18:36:58.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73e5e1e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e019b490 con 0x7f73e00ff430 2023-12-14T18:36:58.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73e5e1e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73e019b650 con 0x7f73e010acf0 2023-12-14T18:36:58.688 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73deffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ffd40 0x7f73e0196880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73deffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ffd40 0x7f73e0196880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50394/0 (socket says 172.21.15.93:50394) 2023-12-14T18:36:58.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.683+0000 7f73deffd700 1 -- 172.21.15.93:0/1013614130 learned_addr learned my addr 172.21.15.93:0/1013614130 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:58.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73dffff700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e010acf0 0x7f73e019ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73df7fe700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73e00ff430 0x7f73e0196340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73deffd700 1 -- 172.21.15.93:0/1013614130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e010acf0 msgr2=0x7f73e019ace0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73deffd700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e010acf0 0x7f73e019ace0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.689 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73deffd700 1 -- 172.21.15.93:0/1013614130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73e00ff430 msgr2=0x7f73e0196340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73deffd700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73e00ff430 0x7f73e0196340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73deffd700 1 -- 172.21.15.93:0/1013614130 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73d40095f0 con 0x7f73e00ffd40 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73df7fe700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73e00ff430 0x7f73e0196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73deffd700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ffd40 0x7f73e0196880 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f73d000e9a0 tx=0x7f73d000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73dcff9700 1 -- 172.21.15.93:0/1013614130 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0009790 con 0x7f73e00ffd40 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73dcff9700 1 -- 172.21.15.93:0/1013614130 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f73d000cbd0 con 0x7f73e00ffd40 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73dcff9700 1 -- 172.21.15.93:0/1013614130 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73d0018680 con 0x7f73e00ffd40 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73e0110350 con 0x7f73e00ffd40 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.684+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73e01101c0 con 0x7f73e00ffd40 2023-12-14T18:36:58.690 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.685+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73e01104e0 con 0x7f73e00ffd40 2023-12-14T18:36:58.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.687+0000 7f73dcff9700 1 -- 172.21.15.93:0/1013614130 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f73d000cd40 con 0x7f73e00ffd40 2023-12-14T18:36:58.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.687+0000 7f73dcff9700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f73c80a1340 0x7f73c80a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:58.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.687+0000 7f73df7fe700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f73c80a1340 0x7f73c80a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:58.691 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.687+0000 7f73dcff9700 1 -- 172.21.15.93:0/1013614130 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7f73d0014070 con 0x7f73e00ffd40 2023-12-14T18:36:58.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.688+0000 7f73df7fe700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f73c80a1340 0x7f73c80a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f73d4005cd0 tx=0x7f73d4005c60 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:58.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.691+0000 7f73dcff9700 1 -- 172.21.15.93:0/1013614130 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f73d0096870 con 0x7f73e00ffd40 2023-12-14T18:36:58.882 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.881+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f73e01a3440 con 0x7f73e00ffd40 2023-12-14T18:36:58.883 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.881+0000 7f73dcff9700 1 -- 172.21.15.93:0/1013614130 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v96) v1 ==== 104+0+11 (secure 0 0 0) 0x7f73d0099e90 con 0x7f73e00ffd40 2023-12-14T18:36:58.883 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:36:58.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f73c80a1340 msgr2=0x7f73c80a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.885 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f73c80a1340 0x7f73c80a37f0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f73d4005cd0 tx=0x7f73d4005c60 comp rx=0 tx=0).stop 2023-12-14T18:36:58.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ffd40 msgr2=0x7f73e0196880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:58.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ffd40 0x7f73e0196880 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f73d000e9a0 tx=0x7f73d000ed60 comp rx=0 tx=0).stop 2023-12-14T18:36:58.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 shutdown_connections 2023-12-14T18:36:58.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f73e010acf0 0x7f73e019ace0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f73e00ff430 0x7f73e0196340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f73c80a1340 0x7f73c80a37f0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 --2- 172.21.15.93:0/1013614130 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f73e00ffd40 0x7f73e0196880 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:58.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 >> 172.21.15.93:0/1013614130 conn(0x7f73e00749c0 msgr2=0x7f73e00fc410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:58.886 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 shutdown_connections 2023-12-14T18:36:58.887 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:58.883+0000 7f73e5e1e700 1 -- 172.21.15.93:0/1013614130 wait complete. 2023-12-14T18:36:58.897 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-14T18:36:59.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.021+0000 7f32f8875700 1 -- 172.21.15.93:0/2210064497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f41004d0 msgr2=0x7f32f41008a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.021+0000 7f32f8875700 1 --2- 172.21.15.93:0/2210064497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f41004d0 0x7f32f41008a0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f32dc0099c0 tx=0x7f32dc009cd0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.021+0000 7f32f8875700 1 -- 172.21.15.93:0/2210064497 shutdown_connections 2023-12-14T18:36:59.023 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.021+0000 7f32f8875700 1 --2- 172.21.15.93:0/2210064497 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f41004d0 0x7f32f41008a0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.021+0000 7f32f8875700 1 --2- 172.21.15.93:0/2210064497 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32f4105510 0x7f32f410d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.021+0000 7f32f8875700 1 --2- 172.21.15.93:0/2210064497 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f4100e70 0x7f32f4104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.021+0000 7f32f8875700 1 -- 172.21.15.93:0/2210064497 >> 172.21.15.93:0/2210064497 conn(0x7f32f4078210 msgr2=0x7f32f4078610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:59.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.021+0000 7f32f8875700 1 -- 172.21.15.93:0/2210064497 shutdown_connections 2023-12-14T18:36:59.024 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.021+0000 7f32f8875700 1 -- 172.21.15.93:0/2210064497 wait complete. 2023-12-14T18:36:59.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f8875700 1 Processor -- start 2023-12-14T18:36:59.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f8875700 1 -- start start 2023-12-14T18:36:59.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f8875700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32f41004d0 0x7f32f41963f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.026 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f8875700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f4100e70 0x7f32f4196930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f8875700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f4105510 0x7f32f419ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f8875700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f419b3d0 con 0x7f32f4100e70 2023-12-14T18:36:59.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f8875700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f419b540 con 0x7f32f4105510 2023-12-14T18:36:59.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f8875700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32f419b700 con 0x7f32f41004d0 2023-12-14T18:36:59.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f4100e70 0x7f32f4196930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f1d9b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f4100e70 0x7f32f4196930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50412/0 (socket says 172.21.15.93:50412) 2023-12-14T18:36:59.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f1d9b700 1 -- 172.21.15.93:0/2229628751 learned_addr learned my addr 172.21.15.93:0/2229628751 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:59.027 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f259c700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32f41004d0 0x7f32f41963f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f2d9d700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f4105510 0x7f32f419ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f1d9b700 1 -- 172.21.15.93:0/2229628751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32f41004d0 msgr2=0x7f32f41963f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f1d9b700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32f41004d0 0x7f32f41963f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f1d9b700 1 -- 172.21.15.93:0/2229628751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f4105510 msgr2=0x7f32f419ad90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f1d9b700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f4105510 0x7f32f419ad90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f1d9b700 1 -- 172.21.15.93:0/2229628751 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f32dc0095f0 con 0x7f32f4100e70 2023-12-14T18:36:59.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.022+0000 7f32f2d9d700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f4105510 0x7f32f419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:59.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.023+0000 7f32f1d9b700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f4100e70 0x7f32f4196930 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f32e400e940 tx=0x7f32e400ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:59.028 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.023+0000 7f32eb7fe700 1 -- 172.21.15.93:0/2229628751 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4009730 con 0x7f32f4100e70 2023-12-14T18:36:59.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.023+0000 7f32eb7fe700 1 -- 172.21.15.93:0/2229628751 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f32e400ee90 con 0x7f32f4100e70 2023-12-14T18:36:59.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.023+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f32f4110320 con 0x7f32f4100e70 2023-12-14T18:36:59.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.023+0000 7f32eb7fe700 1 -- 172.21.15.93:0/2229628751 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32e4010ea0 con 0x7f32f4100e70 2023-12-14T18:36:59.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.023+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f32f4110190 con 0x7f32f4100e70 2023-12-14T18:36:59.029 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.024+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f32f41104b0 con 0x7f32f4100e70 2023-12-14T18:36:59.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.025+0000 7f32eb7fe700 1 -- 172.21.15.93:0/2229628751 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f32e400cbd0 con 0x7f32f4100e70 2023-12-14T18:36:59.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.025+0000 7f32eb7fe700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f32e00a12c0 0x7f32e00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.025+0000 7f32f259c700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f32e00a12c0 0x7f32e00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.025+0000 7f32eb7fe700 1 -- 172.21.15.93:0/2229628751 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7f32e4014070 con 0x7f32f4100e70 2023-12-14T18:36:59.030 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.026+0000 7f32f259c700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f32e00a12c0 0x7f32e00a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009990 tx=0x7f32dc005fb0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:59.031 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.029+0000 7f32eb7fe700 1 -- 172.21.15.93:0/2229628751 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f32e4051730 con 0x7f32f4100e70 2023-12-14T18:36:59.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.218+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f32f41a34c0 con 0x7f32f4100e70 2023-12-14T18:36:59.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.218+0000 7f32eb7fe700 1 -- 172.21.15.93:0/2229628751 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v96) v1 ==== 100+0+11 (secure 0 0 0) 0x7f32e4096340 con 0x7f32f4100e70 2023-12-14T18:36:59.220 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 64 2023-12-14T18:36:59.222 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f32e00a12c0 msgr2=0x7f32e00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f32e00a12c0 0x7f32e00a3770 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f32dc009990 tx=0x7f32dc005fb0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f4100e70 msgr2=0x7f32f4196930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f4100e70 0x7f32f4196930 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f32e400e940 tx=0x7f32e400ec50 comp rx=0 tx=0).stop 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 shutdown_connections 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f32f4105510 0x7f32f419ad90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f32f41004d0 0x7f32f41963f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f32e00a12c0 0x7f32e00a3770 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 --2- 172.21.15.93:0/2229628751 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f32f4100e70 0x7f32f4196930 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.220+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 >> 172.21.15.93:0/2229628751 conn(0x7f32f4078210 msgr2=0x7f32f40fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:59.223 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.221+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 shutdown_connections 2023-12-14T18:36:59.224 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.221+0000 7f32f8875700 1 -- 172.21.15.93:0/2229628751 wait complete. 2023-12-14T18:36:59.233 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:59 smithi093 ceph-mon[42070]: pgmap v327: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:59.234 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:59 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2529327922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:36:59.234 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:59 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1885692237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:36:59.234 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:36:59 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1013614130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:36:59.235 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2023-12-14T18:36:59.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:59 smithi097 ceph-mon[43486]: pgmap v327: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:59.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:59 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2529327922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:36:59.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:59 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1885692237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:36:59.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:36:59 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1013614130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:36:59.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.358+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/1464459680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c841054f0 msgr2=0x7f7c8410d0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.358+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/1464459680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c841054f0 0x7f7c8410d0c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7c7c00b0d0 tx=0x7f7c7c00b3e0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.358+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/1464459680 shutdown_connections 2023-12-14T18:36:59.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.358+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/1464459680 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c841054f0 0x7f7c8410d0c0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.358+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/1464459680 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c841004b0 0x7f7c84100880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.358+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/1464459680 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c84100e50 0x7f7c84104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.358+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/1464459680 >> 172.21.15.93:0/1464459680 conn(0x7f7c840fbd40 msgr2=0x7f7c840fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:59.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.358+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/1464459680 shutdown_connections 2023-12-14T18:36:59.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.358+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/1464459680 wait complete. 2023-12-14T18:36:59.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c8b4ec700 1 Processor -- start 2023-12-14T18:36:59.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c8b4ec700 1 -- start start 2023-12-14T18:36:59.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c8b4ec700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c841004b0 0x7f7c84196450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c8b4ec700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c84100e50 0x7f7c84196990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.363 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c8b4ec700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c841054f0 0x7f7c8419adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c8b4ec700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8419b430 con 0x7f7c84100e50 2023-12-14T18:36:59.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c8b4ec700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8419b5a0 con 0x7f7c841054f0 2023-12-14T18:36:59.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c8b4ec700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c8419b760 con 0x7f7c841004b0 2023-12-14T18:36:59.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c88a87700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c84100e50 0x7f7c84196990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c88a87700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c84100e50 0x7f7c84196990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50428/0 (socket says 172.21.15.93:50428) 2023-12-14T18:36:59.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c88a87700 1 -- 172.21.15.93:0/2198166001 learned_addr learned my addr 172.21.15.93:0/2198166001 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:59.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c89288700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c841004b0 0x7f7c84196450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.364 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c89a89700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c841054f0 0x7f7c8419adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c88a87700 1 -- 172.21.15.93:0/2198166001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c841004b0 msgr2=0x7f7c84196450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c88a87700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c841004b0 0x7f7c84196450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c88a87700 1 -- 172.21.15.93:0/2198166001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c841054f0 msgr2=0x7f7c8419adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c88a87700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c841054f0 0x7f7c8419adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.359+0000 7f7c88a87700 1 -- 172.21.15.93:0/2198166001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7c009d00 con 0x7f7c84100e50 2023-12-14T18:36:59.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.360+0000 7f7c88a87700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c84100e50 0x7f7c84196990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400c9b0 tx=0x7f7c7400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:59.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.360+0000 7f7c7a7fc700 1 -- 172.21.15.93:0/2198166001 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74007aa0 con 0x7f7c84100e50 2023-12-14T18:36:59.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.360+0000 7f7c7a7fc700 1 -- 172.21.15.93:0/2198166001 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f7c74007c00 con 0x7f7c84100e50 2023-12-14T18:36:59.365 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.360+0000 7f7c7a7fc700 1 -- 172.21.15.93:0/2198166001 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c74018660 con 0x7f7c84100e50 2023-12-14T18:36:59.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.360+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c841102c0 con 0x7f7c84100e50 2023-12-14T18:36:59.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.360+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c84110130 con 0x7f7c84100e50 2023-12-14T18:36:59.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.361+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c84110450 con 0x7f7c84100e50 2023-12-14T18:36:59.366 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.362+0000 7f7c7a7fc700 1 -- 172.21.15.93:0/2198166001 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f7c7400f450 con 0x7f7c84100e50 2023-12-14T18:36:59.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.363+0000 7f7c7a7fc700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7c700a12c0 0x7f7c700a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.363+0000 7f7c89288700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7c700a12c0 0x7f7c700a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.363+0000 7f7c7a7fc700 1 -- 172.21.15.93:0/2198166001 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7f7c740cd2d0 con 0x7f7c84100e50 2023-12-14T18:36:59.367 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.363+0000 7f7c89288700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7c700a12c0 0x7f7c700a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005950 tx=0x7f7c8000b410 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:59.368 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.366+0000 7f7c7a7fc700 1 -- 172.21.15.93:0/2198166001 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7c740963c0 con 0x7f7c84100e50 2023-12-14T18:36:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:59 smithi160 ceph-mon[43840]: pgmap v327: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s rd, 4 op/s 2023-12-14T18:36:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:59 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2529327922' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:36:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:59 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1885692237' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2023-12-14T18:36:59.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:36:59 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1013614130' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2023-12-14T18:36:59.558 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.557+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) v1 -- 0x7f7c841a3520 con 0x7f7c84100e50 2023-12-14T18:36:59.559 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.557+0000 7f7c7a7fc700 1 -- 172.21.15.93:0/2198166001 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v96) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7c740999e0 con 0x7f7c84100e50 2023-12-14T18:36:59.559 INFO:teuthology.orchestra.run.smithi093.stdout:pg_num: 32 2023-12-14T18:36:59.561 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7c700a12c0 msgr2=0x7f7c700a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.561 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7c700a12c0 0x7f7c700a3770 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f7c80005950 tx=0x7f7c8000b410 comp rx=0 tx=0).stop 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c84100e50 msgr2=0x7f7c84196990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c84100e50 0x7f7c84196990 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f7c7400c9b0 tx=0x7f7c7400ccc0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 shutdown_connections 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7c841054f0 0x7f7c8419adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7c841004b0 0x7f7c84196450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f7c700a12c0 0x7f7c700a3770 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 --2- 172.21.15.93:0/2198166001 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7c84100e50 0x7f7c84196990 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 >> 172.21.15.93:0/2198166001 conn(0x7f7c840fbd40 msgr2=0x7f7c840fc790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 shutdown_connections 2023-12-14T18:36:59.562 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.559+0000 7f7c8b4ec700 1 -- 172.21.15.93:0/2198166001 wait complete. 2023-12-14T18:36:59.574 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-14T18:36:59.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 -- 172.21.15.93:0/2984101388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2894101360 msgr2=0x7f2894101730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 --2- 172.21.15.93:0/2984101388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2894101360 0x7f2894101730 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f2884009990 tx=0x7f2884009ca0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 -- 172.21.15.93:0/2984101388 shutdown_connections 2023-12-14T18:36:59.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 --2- 172.21.15.93:0/2984101388 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f2894101360 0x7f2894101730 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 --2- 172.21.15.93:0/2984101388 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f289410a880 0x7f289410cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 --2- 172.21.15.93:0/2984101388 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2894101c70 0x7f289410a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 -- 172.21.15.93:0/2984101388 >> 172.21.15.93:0/2984101388 conn(0x7f2894075fd0 msgr2=0x7f28940763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:59.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 -- 172.21.15.93:0/2984101388 shutdown_connections 2023-12-14T18:36:59.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 -- 172.21.15.93:0/2984101388 wait complete. 2023-12-14T18:36:59.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 Processor -- start 2023-12-14T18:36:59.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.695+0000 7f289c39e700 1 -- start start 2023-12-14T18:36:59.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289c39e700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2894101360 0x7f2894196330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289c39e700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2894101c70 0x7f2894196870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289c39e700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289410a880 0x7f289419acd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289c39e700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289419b310 con 0x7f289410a880 2023-12-14T18:36:59.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289c39e700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289419b480 con 0x7f2894101360 2023-12-14T18:36:59.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289c39e700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f289419b640 con 0x7f2894101c70 2023-12-14T18:36:59.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a93b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289410a880 0x7f289419acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a93b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289410a880 0x7f289419acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50438/0 (socket says 172.21.15.93:50438) 2023-12-14T18:36:59.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a93b700 1 -- 172.21.15.93:0/1021278580 learned_addr learned my addr 172.21.15.93:0/1021278580 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:36:59.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a13a700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2894101360 0x7f2894196330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f2899939700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2894101c70 0x7f2894196870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a93b700 1 -- 172.21.15.93:0/1021278580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2894101c70 msgr2=0x7f2894196870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a93b700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2894101c70 0x7f2894196870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a93b700 1 -- 172.21.15.93:0/1021278580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2894101360 msgr2=0x7f2894196330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a93b700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2894101360 0x7f2894196330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a93b700 1 -- 172.21.15.93:0/1021278580 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28840095f0 con 0x7f289410a880 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a13a700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2894101360 0x7f2894196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289a93b700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289410a880 0x7f289419acd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f288c007ee0 tx=0x7f288c00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f288b7fe700 1 -- 172.21.15.93:0/1021278580 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c017070 con 0x7f289410a880 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f288b7fe700 1 -- 172.21.15.93:0/1021278580 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f288c00f040 con 0x7f289410a880 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28941102f0 con 0x7f289410a880 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f288b7fe700 1 -- 172.21.15.93:0/1021278580 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f288c004650 con 0x7f289410a880 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.696+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2894110160 con 0x7f289410a880 2023-12-14T18:36:59.702 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.697+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2894110480 con 0x7f289410a880 2023-12-14T18:36:59.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.698+0000 7f288b7fe700 1 -- 172.21.15.93:0/1021278580 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f288c00dcd0 con 0x7f289410a880 2023-12-14T18:36:59.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.699+0000 7f288b7fe700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f28800a12c0 0x7f28800a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:36:59.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.699+0000 7f289a13a700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f28800a12c0 0x7f28800a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:36:59.703 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.699+0000 7f288b7fe700 1 -- 172.21.15.93:0/1021278580 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7f288c0cc750 con 0x7f289410a880 2023-12-14T18:36:59.704 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.699+0000 7f289a13a700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f28800a12c0 0x7f28800a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2884006010 tx=0x7f288400b540 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:36:59.705 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.703+0000 7f288b7fe700 1 -- 172.21.15.93:0/1021278580 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f288c095840 con 0x7f289410a880 2023-12-14T18:36:59.908 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.906+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f2894067160 con 0x7f289410a880 2023-12-14T18:36:59.916 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.906+0000 7f288b7fe700 1 -- 172.21.15.93:0/1021278580 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v96) v1 ==== 81+0+942 (secure 0 0 0) 0x7f288c098e60 con 0x7f289410a880 2023-12-14T18:36:59.917 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1584909638 2023-12-15T18:26:27.742320+0000 2023-12-14T18:36:59.917 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1420703523 2023-12-15T18:26:27.742320+0000 2023-12-14T18:36:59.917 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/932543927 2023-12-15T18:26:27.742320+0000 2023-12-14T18:36:59.917 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/2647964405 2023-12-15T18:19:35.884432+0000 2023-12-14T18:36:59.918 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/3363004642 2023-12-15T18:19:35.884432+0000 2023-12-14T18:36:59.918 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/3802499407 2023-12-15T18:26:27.742320+0000 2023-12-14T18:36:59.918 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/3802499407 2023-12-15T18:26:27.742320+0000 2023-12-14T18:36:59.918 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2583563521 2023-12-15T18:19:15.054604+0000 2023-12-14T18:36:59.918 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/1226958737 2023-12-15T18:19:35.884432+0000 2023-12-14T18:36:59.918 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6800/1555415520 2023-12-15T18:19:15.054604+0000 2023-12-14T18:36:59.918 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/216269584 2023-12-15T18:19:15.054604+0000 2023-12-14T18:36:59.918 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/1555415520 2023-12-15T18:19:15.054604+0000 2023-12-14T18:36:59.919 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/978480379 2023-12-15T18:26:27.742320+0000 2023-12-14T18:36:59.919 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:6801/2647964405 2023-12-15T18:19:35.884432+0000 2023-12-14T18:36:59.919 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/630684095 2023-12-15T18:19:15.054604+0000 2023-12-14T18:36:59.919 INFO:teuthology.orchestra.run.smithi093.stdout:172.21.15.93:0/2376464512 2023-12-15T18:19:35.884432+0000 2023-12-14T18:36:59.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f28800a12c0 msgr2=0x7f28800a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f28800a12c0 0x7f28800a3770 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2884006010 tx=0x7f288400b540 comp rx=0 tx=0).stop 2023-12-14T18:36:59.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289410a880 msgr2=0x7f289419acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:36:59.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289410a880 0x7f289419acd0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f288c007ee0 tx=0x7f288c00d3b0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 shutdown_connections 2023-12-14T18:36:59.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f28800a12c0 0x7f28800a3770 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f289410a880 0x7f289419acd0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.920 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f2894101360 0x7f2894196330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 --2- 172.21.15.93:0/1021278580 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f2894101c70 0x7f2894196870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:36:59.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 >> 172.21.15.93:0/1021278580 conn(0x7f2894075fd0 msgr2=0x7f289410b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:36:59.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 shutdown_connections 2023-12-14T18:36:59.921 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:36:59.908+0000 7f289c39e700 1 -- 172.21.15.93:0/1021278580 wait complete. 2023-12-14T18:36:59.921 INFO:teuthology.orchestra.run.smithi093.stderr:listed 16 entries 2023-12-14T18:36:59.922 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-14T18:36:59.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:36:59.922 DEBUG:teuthology.orchestra.run.smithi093:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-14T18:37:00.010 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:37:00.010 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:37:00.011 DEBUG:teuthology.orchestra.run.smithi093:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-14T18:37:00.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:37:00.112 DEBUG:teuthology.orchestra.run.smithi093:> ip netns list 2023-12-14T18:37:00.272 INFO:teuthology.orchestra.run.smithi093.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-14T18:37:00.273 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:37:00.273 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-14T18:37:00.296 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:00 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2229628751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:37:00.297 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:00 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2198166001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:37:00.297 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:00 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1021278580' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:37:00.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-14T18:37:00.311 DEBUG:teuthology.orchestra.run.smithi093:> sudo ip link delete ceph-brx 2023-12-14T18:37:00.342 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:00 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2229628751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:37:00.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:00 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2198166001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:37:00.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:00 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1021278580' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:37:00.416 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2023-12-14T18:37:00.426 INFO:tasks.cephadm:Teardown begin 2023-12-14T18:37:00.426 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-14T18:37:00.426 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:37:00.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:00 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2229628751' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2023-12-14T18:37:00.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:00 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2198166001' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2023-12-14T18:37:00.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:00 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1021278580' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2023-12-14T18:37:00.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.568+0000 7f4ef14c4700 1 -- 172.21.15.93:0/3305361116 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec100580 msgr2=0x7f4eec103e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:00.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.568+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/3305361116 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec100580 0x7f4eec103e50 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f4ed40099c0 tx=0x7f4ed4009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.568+0000 7f4ef14c4700 1 -- 172.21.15.93:0/3305361116 shutdown_connections 2023-12-14T18:37:00.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.568+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/3305361116 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec104530 0x7f4eec10c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.568+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/3305361116 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eec0ffbe0 0x7f4eec0fffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.568+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/3305361116 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec100580 0x7f4eec103e50 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.571 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.568+0000 7f4ef14c4700 1 -- 172.21.15.93:0/3305361116 >> 172.21.15.93:0/3305361116 conn(0x7f4eec074670 msgr2=0x7f4eec074a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:00.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.569+0000 7f4ef14c4700 1 -- 172.21.15.93:0/3305361116 shutdown_connections 2023-12-14T18:37:00.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.569+0000 7f4ef14c4700 1 -- 172.21.15.93:0/3305361116 wait complete. 2023-12-14T18:37:00.572 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.569+0000 7f4ef14c4700 1 Processor -- start 2023-12-14T18:37:00.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.569+0000 7f4ef14c4700 1 -- start start 2023-12-14T18:37:00.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.569+0000 7f4ef14c4700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eec0ffbe0 0x7f4eec196540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:00.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.569+0000 7f4ef14c4700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec100580 0x7f4eec196a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:00.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.569+0000 7f4ef14c4700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec104530 0x7f4eec19aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:00.574 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4ef14c4700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19b520 con 0x7f4eec104530 2023-12-14T18:37:00.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4ef14c4700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19b690 con 0x7f4eec100580 2023-12-14T18:37:00.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4ef14c4700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4eec19b850 con 0x7f4eec0ffbe0 2023-12-14T18:37:00.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec104530 0x7f4eec19aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:00.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeb7fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec104530 0x7f4eec19aee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50450/0 (socket says 172.21.15.93:50450) 2023-12-14T18:37:00.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeb7fe700 1 -- 172.21.15.93:0/7931334 learned_addr learned my addr 172.21.15.93:0/7931334 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:37:00.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eea7fc700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec100580 0x7f4eec196a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:00.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeaffd700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eec0ffbe0 0x7f4eec196540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:00.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeb7fe700 1 -- 172.21.15.93:0/7931334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eec0ffbe0 msgr2=0x7f4eec196540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:00.575 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeb7fe700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eec0ffbe0 0x7f4eec196540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeb7fe700 1 -- 172.21.15.93:0/7931334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec100580 msgr2=0x7f4eec196a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:00.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeb7fe700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec100580 0x7f4eec196a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeb7fe700 1 -- 172.21.15.93:0/7931334 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed40095f0 con 0x7f4eec104530 2023-12-14T18:37:00.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4eeb7fe700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec104530 0x7f4eec19aee0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00ef30 tx=0x7f4edc00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:00.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4ee3fff700 1 -- 172.21.15.93:0/7931334 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc018070 con 0x7f4eec104530 2023-12-14T18:37:00.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4eec10f2a0 con 0x7f4eec104530 2023-12-14T18:37:00.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4ee3fff700 1 -- 172.21.15.93:0/7931334 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f4edc00f040 con 0x7f4eec104530 2023-12-14T18:37:00.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4ee3fff700 1 -- 172.21.15.93:0/7931334 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4edc00cdb0 con 0x7f4eec104530 2023-12-14T18:37:00.576 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.570+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4eec10f110 con 0x7f4eec104530 2023-12-14T18:37:00.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.571+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4eec10f430 con 0x7f4eec104530 2023-12-14T18:37:00.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.572+0000 7f4ee3fff700 1 -- 172.21.15.93:0/7931334 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f4edc003a00 con 0x7f4eec104530 2023-12-14T18:37:00.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.573+0000 7f4ee3fff700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f4ed80a12c0 0x7f4ed80a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:00.577 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.573+0000 7f4ee3fff700 1 -- 172.21.15.93:0/7931334 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7f4edc0cd570 con 0x7f4eec104530 2023-12-14T18:37:00.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.573+0000 7f4eeaffd700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f4ed80a12c0 0x7f4ed80a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:00.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.573+0000 7f4eeaffd700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f4ed80a12c0 0x7f4ed80a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009bf0 tx=0x7f4ee4009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:00.578 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.576+0000 7f4ee3fff700 1 -- 172.21.15.93:0/7931334 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f4edc0966f0 con 0x7f4eec104530 2023-12-14T18:37:00.821 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.819+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4eec067160 con 0x7f4eec104530 2023-12-14T18:37:00.822 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.820+0000 7f4ee3fff700 1 -- 172.21.15.93:0/7931334 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10799 (secure 0 0 0) 0x7f4edc099d10 con 0x7f4eec104530 2023-12-14T18:37:00.822 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:37:00.823 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:36:02.001331+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:37:00.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f4ed80a12c0 msgr2=0x7f4ed80a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:00.824 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f4ed80a12c0 0x7f4ed80a3770 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ee4009bf0 tx=0x7f4ee4009260 comp rx=0 tx=0).stop 2023-12-14T18:37:00.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec104530 msgr2=0x7f4eec19aee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:00.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec104530 0x7f4eec19aee0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f4edc00ef30 tx=0x7f4edc00c3a0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 shutdown_connections 2023-12-14T18:37:00.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f4ed80a12c0 0x7f4ed80a3770 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.825 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f4eec104530 0x7f4eec19aee0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f4eec0ffbe0 0x7f4eec196540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 --2- 172.21.15.93:0/7931334 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f4eec100580 0x7f4eec196a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 >> 172.21.15.93:0/7931334 conn(0x7f4eec074670 msgr2=0x7f4eec0fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:00.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 shutdown_connections 2023-12-14T18:37:00.826 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.822+0000 7f4ef14c4700 1 -- 172.21.15.93:0/7931334 wait complete. 2023-12-14T18:37:00.827 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 36 2023-12-14T18:37:00.836 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-14T18:37:00.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.961+0000 7f6cc202d700 1 -- 172.21.15.93:0/750354042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101430 msgr2=0x7f6cbc101800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:00.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.961+0000 7f6cc202d700 1 --2- 172.21.15.93:0/750354042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101430 0x7f6cbc101800 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f6cac009990 tx=0x7f6cac009ca0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.966 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.961+0000 7f6cc202d700 1 -- 172.21.15.93:0/750354042 shutdown_connections 2023-12-14T18:37:00.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.961+0000 7f6cc202d700 1 --2- 172.21.15.93:0/750354042 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101430 0x7f6cbc101800 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.961+0000 7f6cc202d700 1 --2- 172.21.15.93:0/750354042 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cbc10a770 0x7f6cbc10cc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.961+0000 7f6cc202d700 1 --2- 172.21.15.93:0/750354042 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cbc101d40 0x7f6cbc10a230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.961+0000 7f6cc202d700 1 -- 172.21.15.93:0/750354042 >> 172.21.15.93:0/750354042 conn(0x7f6cbc076370 msgr2=0x7f6cbc076770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:00.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.961+0000 7f6cc202d700 1 -- 172.21.15.93:0/750354042 shutdown_connections 2023-12-14T18:37:00.967 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.961+0000 7f6cc202d700 1 -- 172.21.15.93:0/750354042 wait complete. 2023-12-14T18:37:00.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cc202d700 1 Processor -- start 2023-12-14T18:37:00.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cc202d700 1 -- start start 2023-12-14T18:37:00.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cc202d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cbc101430 0x7f6cbc196380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:00.968 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cc202d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101d40 0x7f6cbc1968c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:00.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cc202d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cbc10a770 0x7f6cbc19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:00.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cc202d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc19b360 con 0x7f6cbc101d40 2023-12-14T18:37:00.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cc202d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc19b4d0 con 0x7f6cbc101430 2023-12-14T18:37:00.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cc202d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cbc19b690 con 0x7f6cbc10a770 2023-12-14T18:37:00.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cbaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101d40 0x7f6cbc1968c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:00.969 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cbbfff700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cbc10a770 0x7f6cbc19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:00.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cbb7fe700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cbc101430 0x7f6cbc196380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:00.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cbaffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101d40 0x7f6cbc1968c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:50474/0 (socket says 172.21.15.93:50474) 2023-12-14T18:37:00.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.962+0000 7f6cbaffd700 1 -- 172.21.15.93:0/2956050415 learned_addr learned my addr 172.21.15.93:0/2956050415 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:37:00.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cbb7fe700 1 -- 172.21.15.93:0/2956050415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cbc10a770 msgr2=0x7f6cbc19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:00.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cbb7fe700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cbc10a770 0x7f6cbc19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.970 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cbb7fe700 1 -- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101d40 msgr2=0x7f6cbc1968c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:00.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cbb7fe700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101d40 0x7f6cbc1968c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:00.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cbb7fe700 1 -- 172.21.15.93:0/2956050415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cac0095f0 con 0x7f6cbc101430 2023-12-14T18:37:00.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cbaffd700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101d40 0x7f6cbc1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:00.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cbbfff700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cbc10a770 0x7f6cbc19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-14T18:37:00.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cbb7fe700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cbc101430 0x7f6cbc196380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6cac000c00 tx=0x7f6cac009580 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:00.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cb8ff9700 1 -- 172.21.15.93:0/2956050415 <== mon.2 v2:172.21.15.97:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac01d070 con 0x7f6cbc101430 2023-12-14T18:37:00.971 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cb8ff9700 1 -- 172.21.15.93:0/2956050415 <== mon.2 v2:172.21.15.97:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f6cac022470 con 0x7f6cbc101430 2023-12-14T18:37:00.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cbc110230 con 0x7f6cbc101430 2023-12-14T18:37:00.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cb8ff9700 1 -- 172.21.15.93:0/2956050415 <== mon.2 v2:172.21.15.97:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cac017650 con 0x7f6cbc101430 2023-12-14T18:37:00.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.963+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cbc1100a0 con 0x7f6cbc101430 2023-12-14T18:37:00.972 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.964+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cbc1103c0 con 0x7f6cbc101430 2023-12-14T18:37:00.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.966+0000 7f6cb8ff9700 1 -- 172.21.15.93:0/2956050415 <== mon.2 v2:172.21.15.97:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f6cac00fb10 con 0x7f6cbc101430 2023-12-14T18:37:00.973 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.966+0000 7f6cb8ff9700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6ca40a1340 0x7f6ca40a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:00.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.967+0000 7f6cb8ff9700 1 -- 172.21.15.93:0/2956050415 <== mon.2 v2:172.21.15.97:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7f6cac0ce160 con 0x7f6cbc101430 2023-12-14T18:37:00.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.967+0000 7f6cbaffd700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6ca40a1340 0x7f6ca40a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:00.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.967+0000 7f6cbaffd700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6ca40a1340 0x7f6ca40a37f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc197990 tx=0x7f6cb0009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:00.974 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:00.971+0000 7f6cb8ff9700 1 -- 172.21.15.93:0/2956050415 <== mon.2 v2:172.21.15.97:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f6cac05c730 con 0x7f6cbc101430 2023-12-14T18:37:01.214 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.212+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6cbc0679f0 con 0x7f6cbc101430 2023-12-14T18:37:01.215 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.213+0000 7f6cb8ff9700 1 -- 172.21.15.93:0/2956050415 <== mon.2 v2:172.21.15.97:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) v1 ==== 94+0+10799 (secure 0 0 0) 0x7f6cac097280 con 0x7f6cbc101430 2023-12-14T18:37:01.216 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:37:01.216 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14550,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6835/53835883","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6834","nonce":53835883},{"type":"v1","addr":"172.21.15.93:6835","nonce":53835883}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":4},{"gid":14562,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6837/1107020110","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6836","nonce":1107020110},{"type":"v1","addr":"172.21.15.93:6837","nonce":1107020110}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":5},{"gid":14607,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6839/1450887105","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6838","nonce":1450887105},{"type":"v1","addr":"172.21.15.93:6839","nonce":1450887105}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":14619,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.93:6841/4265710188","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6840","nonce":4265710188},{"type":"v1","addr":"172.21.15.93:6841","nonce":4265710188}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.160:6833/2184947052","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6832","nonce":2184947052},{"type":"v1","addr":"172.21.15.160:6833","nonce":2184947052}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.97:6833/1737809375","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6832","nonce":1737809375},{"type":"v1","addr":"172.21.15.97:6833","nonce":1737809375}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2023-12-14T18:24:36.557875+0000","modified":"2023-12-14T18:36:02.001331+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":3,"in":[0,1,2],"up":{"mds_0":24461,"mds_1":24442,"mds_2":24422},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24415":{"gid":24415,"name":"f","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6835/2285100381","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6834","nonce":2285100381},{"type":"v1","addr":"172.21.15.160:6835","nonce":2285100381}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24422":{"gid":24422,"name":"e","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.97:6835/1146986690","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6834","nonce":1146986690},{"type":"v1","addr":"172.21.15.97:6835","nonce":1146986690}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24424":{"gid":24424,"name":"c","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.160:6837/233011976","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6836","nonce":233011976},{"type":"v1","addr":"172.21.15.160:6837","nonce":233011976}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24440":{"gid":24440,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.97:6837/3050538896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6836","nonce":3050538896},{"type":"v1","addr":"172.21.15.97:6837","nonce":3050538896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24442":{"gid":24442,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":10,"addr":"172.21.15.160:6839/3370815452","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6838","nonce":3370815452},{"type":"v1","addr":"172.21.15.160:6839","nonce":3370815452}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}},"gid_24461":{"gid":24461,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.97:6839/3753662808","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6838","nonce":3753662808},{"type":"v1","addr":"172.21.15.97:6839","nonce":3753662808}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-14T18:37:01.218 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6ca40a1340 msgr2=0x7f6ca40a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6ca40a1340 0x7f6ca40a37f0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6cbc197990 tx=0x7f6cb0009260 comp rx=0 tx=0).stop 2023-12-14T18:37:01.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cbc101430 msgr2=0x7f6cbc196380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cbc101430 0x7f6cbc196380 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6cac000c00 tx=0x7f6cac009580 comp rx=0 tx=0).stop 2023-12-14T18:37:01.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 shutdown_connections 2023-12-14T18:37:01.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f6cbc10a770 0x7f6cbc19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.219 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f6cbc101430 0x7f6cbc196380 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f6ca40a1340 0x7f6ca40a37f0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 --2- 172.21.15.93:0/2956050415 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f6cbc101d40 0x7f6cbc1968c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.215+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 >> 172.21.15.93:0/2956050415 conn(0x7f6cbc076370 msgr2=0x7f6cbc1021d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:01.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.216+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 shutdown_connections 2023-12-14T18:37:01.220 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.216+0000 7f6cc202d700 1 -- 172.21.15.93:0/2956050415 wait complete. 2023-12-14T18:37:01.220 INFO:teuthology.orchestra.run.smithi093.stderr:dumped fsmap epoch 36 2023-12-14T18:37:01.229 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:01 smithi093 ceph-mon[42070]: pgmap v328: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:37:01.229 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:01 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/7931334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:01.230 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-14T18:37:01.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:01 smithi097 ceph-mon[43486]: pgmap v328: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:37:01.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:01 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/7931334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:01.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.351+0000 7fdbadf78700 1 -- 172.21.15.93:0/3012789913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 msgr2=0x7fdba810c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.351+0000 7fdbadf78700 1 --2- 172.21.15.93:0/3012789913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 0x7fdba810c780 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c009990 tx=0x7fdb9c009ca0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.351+0000 7fdbadf78700 1 -- 172.21.15.93:0/3012789913 shutdown_connections 2023-12-14T18:37:01.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.351+0000 7fdbadf78700 1 --2- 172.21.15.93:0/3012789913 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdba810ccc0 0x7fdba810f0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.351+0000 7fdbadf78700 1 --2- 172.21.15.93:0/3012789913 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdba80690b0 0x7fdba8069480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.356 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.351+0000 7fdbadf78700 1 --2- 172.21.15.93:0/3012789913 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 0x7fdba810c780 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.351+0000 7fdbadf78700 1 -- 172.21.15.93:0/3012789913 >> 172.21.15.93:0/3012789913 conn(0x7fdba8076100 msgr2=0x7fdba8076500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:01.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.351+0000 7fdbadf78700 1 -- 172.21.15.93:0/3012789913 shutdown_connections 2023-12-14T18:37:01.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.351+0000 7fdbadf78700 1 -- 172.21.15.93:0/3012789913 wait complete. 2023-12-14T18:37:01.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdbadf78700 1 Processor -- start 2023-12-14T18:37:01.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdbadf78700 1 -- start start 2023-12-14T18:37:01.357 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdbadf78700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdba80690b0 0x7fdba8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:01.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdbadf78700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 0x7fdba8198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:01.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdbadf78700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdba810ccc0 0x7fdba819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:01.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdbadf78700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba819d600 con 0x7fdba80699c0 2023-12-14T18:37:01.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdbadf78700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba819d770 con 0x7fdba80690b0 2023-12-14T18:37:01.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdbadf78700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdba819d930 con 0x7fdba810ccc0 2023-12-14T18:37:01.358 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 0x7fdba8198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:01.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba6ffd700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 0x7fdba8198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45608/0 (socket says 172.21.15.93:45608) 2023-12-14T18:37:01.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba6ffd700 1 -- 172.21.15.93:0/2692854174 learned_addr learned my addr 172.21.15.93:0/2692854174 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:37:01.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba77fe700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdba80690b0 0x7fdba8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:01.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba7fff700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdba810ccc0 0x7fdba819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:01.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba6ffd700 1 -- 172.21.15.93:0/2692854174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdba810ccc0 msgr2=0x7fdba819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba6ffd700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdba810ccc0 0x7fdba819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba6ffd700 1 -- 172.21.15.93:0/2692854174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdba80690b0 msgr2=0x7fdba8198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.359 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba6ffd700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdba80690b0 0x7fdba8198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba6ffd700 1 -- 172.21.15.93:0/2692854174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb9c0095f0 con 0x7fdba80699c0 2023-12-14T18:37:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba77fe700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdba80690b0 0x7fdba8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.352+0000 7fdba7fff700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdba810ccc0 0x7fdba819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.353+0000 7fdba6ffd700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 0x7fdba8198b60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005910 tx=0x7fdb9c00b610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.353+0000 7fdba4ff9700 1 -- 172.21.15.93:0/2692854174 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c01d070 con 0x7fdba80699c0 2023-12-14T18:37:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.353+0000 7fdba4ff9700 1 -- 172.21.15.93:0/2692854174 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fdb9c00fb80 con 0x7fdba80699c0 2023-12-14T18:37:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.353+0000 7fdba4ff9700 1 -- 172.21.15.93:0/2692854174 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb9c017650 con 0x7fdba80699c0 2023-12-14T18:37:01.360 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.353+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdba81124c0 con 0x7fdba80699c0 2023-12-14T18:37:01.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.353+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdba8112330 con 0x7fdba80699c0 2023-12-14T18:37:01.361 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.354+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdba8112650 con 0x7fdba80699c0 2023-12-14T18:37:01.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.355+0000 7fdba4ff9700 1 -- 172.21.15.93:0/2692854174 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fdb9c004830 con 0x7fdba80699c0 2023-12-14T18:37:01.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.355+0000 7fdba4ff9700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdb900a12c0 0x7fdb900a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:01.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.355+0000 7fdba4ff9700 1 -- 172.21.15.93:0/2692854174 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7fdb9c0ceb90 con 0x7fdba80699c0 2023-12-14T18:37:01.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.355+0000 7fdba77fe700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdb900a12c0 0x7fdb900a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:01.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.356+0000 7fdba77fe700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdb900a12c0 0x7fdb900a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009cb0 tx=0x7fdb98009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:01.362 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.359+0000 7fdba4ff9700 1 -- 172.21.15.93:0/2692854174 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fdb9c097d10 con 0x7fdba80699c0 2023-12-14T18:37:01.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:01 smithi160 ceph-mon[43840]: pgmap v328: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 7 op/s 2023-12-14T18:37:01.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:01 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/7931334' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:01.551 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.549+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdba8067e40 con 0x7fdba80699c0 2023-12-14T18:37:01.553 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.551+0000 7fdba4ff9700 1 -- 172.21.15.93:0/2692854174 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v96) v1 ==== 74+0+24846 (secure 0 0 0) 0x7fdb9c09b330 con 0x7fdba80699c0 2023-12-14T18:37:01.553 INFO:teuthology.orchestra.run.smithi093.stdout: 2023-12-14T18:37:01.553 INFO:teuthology.orchestra.run.smithi093.stdout:{"epoch":96,"fsid":"1c5ac00e-9aad-11ee-95a5-87774f69a715","created":"2023-12-14T18:18:47.086099+0000","modified":"2023-12-14T18:36:40.942497+0000","last_up_change":"2023-12-14T18:24:03.123463+0000","last_in_change":"2023-12-14T18:23:46.892267+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-14T18:21:24.200509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-14T18:24:34.407045+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-14T18:24:35.728548+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2023-12-14T18:24:38.796189+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"94","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":94,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"f4bece78-53dc-4e27-bb2b-32a16baa8936","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6802","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6803","nonce":822854131}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6804","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6805","nonce":822854131}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6808","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6809","nonce":822854131}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6806","nonce":822854131},{"type":"v1","addr":"172.21.15.93:6807","nonce":822854131}]},"public_addr":"172.21.15.93:6803/822854131","cluster_addr":"172.21.15.93:6805/822854131","heartbeat_back_addr":"172.21.15.93:6809/822854131","heartbeat_front_addr":"172.21.15.93:6807/822854131","state":["exists","up"]},{"osd":1,"uuid":"a2968fe6-d55f-4ae4-a2bc-e06b379812ae","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6800","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6801","nonce":3276231390}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6802","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6803","nonce":3276231390}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6806","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6807","nonce":3276231390}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6804","nonce":3276231390},{"type":"v1","addr":"172.21.15.97:6805","nonce":3276231390}]},"public_addr":"172.21.15.97:6801/3276231390","cluster_addr":"172.21.15.97:6803/3276231390","heartbeat_back_addr":"172.21.15.97:6807/3276231390","heartbeat_front_addr":"172.21.15.97:6805/3276231390","state":["exists","up"]},{"osd":2,"uuid":"a9138f66-3120-4842-af4c-40a6c4c57710","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6800","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6801","nonce":3346159121}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6802","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6803","nonce":3346159121}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6806","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6807","nonce":3346159121}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6804","nonce":3346159121},{"type":"v1","addr":"172.21.15.160:6805","nonce":3346159121}]},"public_addr":"172.21.15.160:6801/3346159121","cluster_addr":"172.21.15.160:6803/3346159121","heartbeat_back_addr":"172.21.15.160:6807/3346159121","heartbeat_front_addr":"172.21.15.160:6805/3346159121","state":["exists","up"]},{"osd":3,"uuid":"317f1a16-4f77-46e5-aa43-63e8d930682b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":24,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6810","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6811","nonce":980379303}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6812","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6813","nonce":980379303}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6816","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6817","nonce":980379303}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6814","nonce":980379303},{"type":"v1","addr":"172.21.15.93:6815","nonce":980379303}]},"public_addr":"172.21.15.93:6811/980379303","cluster_addr":"172.21.15.93:6813/980379303","heartbeat_back_addr":"172.21.15.93:6817/980379303","heartbeat_front_addr":"172.21.15.93:6815/980379303","state":["exists","up"]},{"osd":4,"uuid":"7afe6ea6-e5ce-4fea-8e55-a5fbcba54181","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6808","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6809","nonce":3095116912}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6810","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6811","nonce":3095116912}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6814","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6815","nonce":3095116912}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6812","nonce":3095116912},{"type":"v1","addr":"172.21.15.97:6813","nonce":3095116912}]},"public_addr":"172.21.15.97:6809/3095116912","cluster_addr":"172.21.15.97:6811/3095116912","heartbeat_back_addr":"172.21.15.97:6815/3095116912","heartbeat_front_addr":"172.21.15.97:6813/3095116912","state":["exists","up"]},{"osd":5,"uuid":"6882d38d-40ad-40d7-aeb3-8c1e9367ef16","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6808","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6809","nonce":1570120079}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6810","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6811","nonce":1570120079}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6814","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6815","nonce":1570120079}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6812","nonce":1570120079},{"type":"v1","addr":"172.21.15.160:6813","nonce":1570120079}]},"public_addr":"172.21.15.160:6809/1570120079","cluster_addr":"172.21.15.160:6811/1570120079","heartbeat_back_addr":"172.21.15.160:6815/1570120079","heartbeat_front_addr":"172.21.15.160:6813/1570120079","state":["exists","up"]},{"osd":6,"uuid":"4f046b70-3c19-4dac-99f8-2c72520268be","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6818","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6819","nonce":848954134}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6820","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6821","nonce":848954134}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6824","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6825","nonce":848954134}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6822","nonce":848954134},{"type":"v1","addr":"172.21.15.93:6823","nonce":848954134}]},"public_addr":"172.21.15.93:6819/848954134","cluster_addr":"172.21.15.93:6821/848954134","heartbeat_back_addr":"172.21.15.93:6825/848954134","heartbeat_front_addr":"172.21.15.93:6823/848954134","state":["exists","up"]},{"osd":7,"uuid":"527a2a02-7c78-4572-9493-54af215f8c7e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6816","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6817","nonce":1426225102}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6818","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6819","nonce":1426225102}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6822","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6823","nonce":1426225102}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6820","nonce":1426225102},{"type":"v1","addr":"172.21.15.97:6821","nonce":1426225102}]},"public_addr":"172.21.15.97:6817/1426225102","cluster_addr":"172.21.15.97:6819/1426225102","heartbeat_back_addr":"172.21.15.97:6823/1426225102","heartbeat_front_addr":"172.21.15.97:6821/1426225102","state":["exists","up"]},{"osd":8,"uuid":"5d79000b-a942-43b7-85f7-620e6b98bf90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":55,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6816","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6817","nonce":3984975827}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6818","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6819","nonce":3984975827}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6822","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6823","nonce":3984975827}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6820","nonce":3984975827},{"type":"v1","addr":"172.21.15.160:6821","nonce":3984975827}]},"public_addr":"172.21.15.160:6817/3984975827","cluster_addr":"172.21.15.160:6819/3984975827","heartbeat_back_addr":"172.21.15.160:6823/3984975827","heartbeat_front_addr":"172.21.15.160:6821/3984975827","state":["exists","up"]},{"osd":9,"uuid":"14e038ed-c6e1-4d19-88cc-329ffe2343e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":62,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6826","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6827","nonce":4200476017}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6828","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6829","nonce":4200476017}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6832","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6833","nonce":4200476017}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.93:6830","nonce":4200476017},{"type":"v1","addr":"172.21.15.93:6831","nonce":4200476017}]},"public_addr":"172.21.15.93:6827/4200476017","cluster_addr":"172.21.15.93:6829/4200476017","heartbeat_back_addr":"172.21.15.93:6833/4200476017","heartbeat_front_addr":"172.21.15.93:6831/4200476017","state":["exists","up"]},{"osd":10,"uuid":"44df1de1-facd-4592-855a-d32c456cf8ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6824","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6825","nonce":240921879}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6826","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6827","nonce":240921879}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6830","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6831","nonce":240921879}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.97:6828","nonce":240921879},{"type":"v1","addr":"172.21.15.97:6829","nonce":240921879}]},"public_addr":"172.21.15.97:6825/240921879","cluster_addr":"172.21.15.97:6827/240921879","heartbeat_back_addr":"172.21.15.97:6831/240921879","heartbeat_front_addr":"172.21.15.97:6829/240921879","state":["exists","up"]},{"osd":11,"uuid":"82c3b552-1386-4b96-89c6-4263d10a3f2c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":75,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6824","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6825","nonce":747472238}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6826","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6827","nonce":747472238}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6830","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6831","nonce":747472238}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.160:6828","nonce":747472238},{"type":"v1","addr":"172.21.15.160:6829","nonce":747472238}]},"public_addr":"172.21.15.160:6825/747472238","cluster_addr":"172.21.15.160:6827/747472238","heartbeat_back_addr":"172.21.15.160:6831/747472238","heartbeat_front_addr":"172.21.15.160:6829/747472238","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:20:46.746034+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:04.791187+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:22.096961+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:39.779564+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:21:57.769485+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:15.550207+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:32.996648+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:22:50.793211+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:08.054324+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:25.762889+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:23:43.696093+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-14T18:24:01.443507+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.6","mappings":[{"from":8,"to":11}]},{"pgid":"2.a","mappings":[{"from":2,"to":11}]},{"pgid":"2.1b","mappings":[{"from":2,"to":4}]},{"pgid":"3.10","mappings":[{"from":5,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.34","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.3","mappings":[{"from":5,"to":8}]},{"pgid":"4.c","mappings":[{"from":9,"to":10}]},{"pgid":"4.f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.93:0/1584909638":"2023-12-15T18:26:27.742320+0000","172.21.15.93:0/1420703523":"2023-12-15T18:26:27.742320+0000","172.21.15.93:0/932543927":"2023-12-15T18:26:27.742320+0000","172.21.15.93:6800/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/3363004642":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6801/3802499407":"2023-12-15T18:26:27.742320+0000","172.21.15.93:6800/3802499407":"2023-12-15T18:26:27.742320+0000","172.21.15.93:0/2583563521":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/1226958737":"2023-12-15T18:19:35.884432+0000","172.21.15.93:6800/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/216269584":"2023-12-15T18:19:15.054604+0000","172.21.15.93:6801/1555415520":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/978480379":"2023-12-15T18:26:27.742320+0000","172.21.15.93:6801/2647964405":"2023-12-15T18:19:35.884432+0000","172.21.15.93:0/630684095":"2023-12-15T18:19:15.054604+0000","172.21.15.93:0/2376464512":"2023-12-15T18:19:35.884432+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-14T18:37:01.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdb900a12c0 msgr2=0x7fdb900a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdb900a12c0 0x7fdb900a3770 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdb98009cb0 tx=0x7fdb98009260 comp rx=0 tx=0).stop 2023-12-14T18:37:01.555 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 msgr2=0x7fdba8198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 0x7fdba8198b60 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fdb9c005910 tx=0x7fdb9c00b610 comp rx=0 tx=0).stop 2023-12-14T18:37:01.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 shutdown_connections 2023-12-14T18:37:01.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fdba810ccc0 0x7fdba819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fdba80690b0 0x7fdba8198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fdb900a12c0 0x7fdb900a3770 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 --2- 172.21.15.93:0/2692854174 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fdba80699c0 0x7fdba8198b60 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 >> 172.21.15.93:0/2692854174 conn(0x7fdba8076100 msgr2=0x7fdba80fcd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:01.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 shutdown_connections 2023-12-14T18:37:01.556 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.553+0000 7fdbadf78700 1 -- 172.21.15.93:0/2692854174 wait complete. 2023-12-14T18:37:01.568 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-14T18:37:01.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.690+0000 7f24ba911700 1 -- 172.21.15.93:0/1966299887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ff3f0 msgr2=0x7f24b40ff7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.690+0000 7f24ba911700 1 --2- 172.21.15.93:0/1966299887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ff3f0 0x7f24b40ff7c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f24a40099c0 tx=0x7f24a4009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.690+0000 7f24ba911700 1 -- 172.21.15.93:0/1966299887 shutdown_connections 2023-12-14T18:37:01.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.690+0000 7f24ba911700 1 --2- 172.21.15.93:0/1966299887 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ff3f0 0x7f24b40ff7c0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.692 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.690+0000 7f24ba911700 1 --2- 172.21.15.93:0/1966299887 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b4104430 0x7f24b410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.690+0000 7f24ba911700 1 --2- 172.21.15.93:0/1966299887 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b40ffd90 0x7f24b4103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.690+0000 7f24ba911700 1 -- 172.21.15.93:0/1966299887 >> 172.21.15.93:0/1966299887 conn(0x7f24b4075fd0 msgr2=0x7f24b40763d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:01.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.690+0000 7f24ba911700 1 -- 172.21.15.93:0/1966299887 shutdown_connections 2023-12-14T18:37:01.693 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.690+0000 7f24ba911700 1 -- 172.21.15.93:0/1966299887 wait complete. 2023-12-14T18:37:01.695 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24ba911700 1 Processor -- start 2023-12-14T18:37:01.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24ba911700 1 -- start start 2023-12-14T18:37:01.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24ba911700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b40ff3f0 0x7f24b41952d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:01.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24ba911700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ffd90 0x7f24b4195810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:01.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24ba911700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b4104430 0x7f24b4199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:01.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24ba911700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b419a2b0 con 0x7f24b40ffd90 2023-12-14T18:37:01.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24ba911700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b419a420 con 0x7f24b40ff3f0 2023-12-14T18:37:01.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24ba911700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b419a5e0 con 0x7f24b4104430 2023-12-14T18:37:01.696 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24b37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ffd90 0x7f24b4195810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:01.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24b37fe700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ffd90 0x7f24b4195810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45628/0 (socket says 172.21.15.93:45628) 2023-12-14T18:37:01.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.691+0000 7f24b37fe700 1 -- 172.21.15.93:0/295239884 learned_addr learned my addr 172.21.15.93:0/295239884 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:37:01.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b8eae700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b4104430 0x7f24b4199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:01.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b3fff700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b40ff3f0 0x7f24b41952d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:01.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b37fe700 1 -- 172.21.15.93:0/295239884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b4104430 msgr2=0x7f24b4199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b37fe700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b4104430 0x7f24b4199c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b37fe700 1 -- 172.21.15.93:0/295239884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b40ff3f0 msgr2=0x7f24b41952d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:01.697 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b37fe700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b40ff3f0 0x7f24b41952d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:01.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b37fe700 1 -- 172.21.15.93:0/295239884 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a40095f0 con 0x7f24b40ffd90 2023-12-14T18:37:01.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b3fff700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b40ff3f0 0x7f24b41952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-14T18:37:01.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b37fe700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ffd90 0x7f24b4195810 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f249c00d950 tx=0x7f249c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:01.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b17fa700 1 -- 172.21.15.93:0/295239884 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c009b40 con 0x7f24b40ffd90 2023-12-14T18:37:01.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b410f240 con 0x7f24b40ffd90 2023-12-14T18:37:01.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b17fa700 1 -- 172.21.15.93:0/295239884 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f249c00de30 con 0x7f24b40ffd90 2023-12-14T18:37:01.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24b17fa700 1 -- 172.21.15.93:0/295239884 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f249c00f620 con 0x7f24b40ffd90 2023-12-14T18:37:01.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.692+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b410f0b0 con 0x7f24b40ffd90 2023-12-14T18:37:01.698 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.693+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b410f3d0 con 0x7f24b40ffd90 2023-12-14T18:37:01.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.694+0000 7f24b17fa700 1 -- 172.21.15.93:0/295239884 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f249c010460 con 0x7f24b40ffd90 2023-12-14T18:37:01.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.694+0000 7f24b17fa700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f24a00a12c0 0x7f24a00a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:01.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.695+0000 7f24b3fff700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f24a00a12c0 0x7f24a00a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:01.699 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.695+0000 7f24b17fa700 1 -- 172.21.15.93:0/295239884 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 9322+0+0 (secure 0 0 0) 0x7f249c0ccc80 con 0x7f24b40ffd90 2023-12-14T18:37:01.700 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.695+0000 7f24b3fff700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f24a00a12c0 0x7f24a00a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006010 tx=0x7f24a4000bc0 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:01.701 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.700+0000 7f24b17fa700 1 -- 172.21.15.93:0/295239884 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f249c095d10 con 0x7f24b40ffd90 2023-12-14T18:37:01.929 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:01.927+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f24b419aab0 con 0x7f24b40ffd90 2023-12-14T18:37:02.084 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.082+0000 7f24b17fa700 1 -- 172.21.15.93:0/295239884 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v37) v1 ==== 161+0+0 (secure 0 0 0) 0x7f249c099330 con 0x7f24b40ffd90 2023-12-14T18:37:02.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f24a00a12c0 msgr2=0x7f24a00a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:02.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f24a00a12c0 0x7f24a00a3770 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f24a4006010 tx=0x7f24a4000bc0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ffd90 msgr2=0x7f24b4195810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:02.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ffd90 0x7f24b4195810 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f249c00d950 tx=0x7f249c00dd10 comp rx=0 tx=0).stop 2023-12-14T18:37:02.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 shutdown_connections 2023-12-14T18:37:02.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f24b4104430 0x7f24b4199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.087 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f24b40ff3f0 0x7f24b41952d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f24a00a12c0 0x7f24a00a3770 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 --2- 172.21.15.93:0/295239884 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f24b40ffd90 0x7f24b4195810 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 >> 172.21.15.93:0/295239884 conn(0x7f24b4075fd0 msgr2=0x7f24b40fccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:02.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 shutdown_connections 2023-12-14T18:37:02.088 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.085+0000 7f24ba911700 1 -- 172.21.15.93:0/295239884 wait complete. 2023-12-14T18:37:02.088 INFO:teuthology.orchestra.run.smithi093.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-14T18:37:02.099 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-14T18:37:02.231 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.227+0000 7fce1601b700 1 -- 172.21.15.93:0/30544881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce100ff3d0 msgr2=0x7fce100ff7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:02.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.227+0000 7fce1601b700 1 --2- 172.21.15.93:0/30544881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce100ff3d0 0x7fce100ff7a0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fcdf80099c0 tx=0x7fcdf8009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.227+0000 7fce1601b700 1 -- 172.21.15.93:0/30544881 shutdown_connections 2023-12-14T18:37:02.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.227+0000 7fce1601b700 1 --2- 172.21.15.93:0/30544881 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce100ff3d0 0x7fce100ff7a0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.227+0000 7fce1601b700 1 --2- 172.21.15.93:0/30544881 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce10104410 0x7fce1010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.227+0000 7fce1601b700 1 --2- 172.21.15.93:0/30544881 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce100ffd70 0x7fce10103d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.232 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.227+0000 7fce1601b700 1 -- 172.21.15.93:0/30544881 >> 172.21.15.93:0/30544881 conn(0x7fce10076000 msgr2=0x7fce10076400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:02.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.227+0000 7fce1601b700 1 -- 172.21.15.93:0/30544881 shutdown_connections 2023-12-14T18:37:02.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.227+0000 7fce1601b700 1 -- 172.21.15.93:0/30544881 wait complete. 2023-12-14T18:37:02.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce1601b700 1 Processor -- start 2023-12-14T18:37:02.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce1601b700 1 -- start start 2023-12-14T18:37:02.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce1601b700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce100ff3d0 0x7fce10195080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:02.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce1601b700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce100ffd70 0x7fce101955c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:02.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce1601b700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce10104410 0x7fce10195b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:02.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce1601b700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce101a48c0 con 0x7fce10104410 2023-12-14T18:37:02.233 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce1601b700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce101a4a30 con 0x7fce100ffd70 2023-12-14T18:37:02.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce1601b700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce101a4bf0 con 0x7fce100ff3d0 2023-12-14T18:37:02.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce0ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce10104410 0x7fce10195b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:02.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce0ffff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce10104410 0x7fce10195b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45638/0 (socket says 172.21.15.93:45638) 2023-12-14T18:37:02.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce0ffff700 1 -- 172.21.15.93:0/3922475011 learned_addr learned my addr 172.21.15.93:0/3922475011 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:37:02.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce0effd700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce100ffd70 0x7fce101955c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:02.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.228+0000 7fce0f7fe700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce100ff3d0 0x7fce10195080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:02.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0ffff700 1 -- 172.21.15.93:0/3922475011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce100ff3d0 msgr2=0x7fce10195080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:02.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0ffff700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce100ff3d0 0x7fce10195080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.234 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0ffff700 1 -- 172.21.15.93:0/3922475011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce100ffd70 msgr2=0x7fce101955c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:02.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0ffff700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce100ffd70 0x7fce101955c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:02.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0ffff700 1 -- 172.21.15.93:0/3922475011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80095f0 con 0x7fce10104410 2023-12-14T18:37:02.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0effd700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce100ffd70 0x7fce101955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:02.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0f7fe700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce100ff3d0 0x7fce10195080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-14T18:37:02.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0ffff700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce10104410 0x7fce10195b00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fce0800ada0 tx=0x7fce0800c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:02.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0cff9700 1 -- 172.21.15.93:0/3922475011 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce0800f070 con 0x7fce10104410 2023-12-14T18:37:02.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0cff9700 1 -- 172.21.15.93:0/3922475011 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fce08007b30 con 0x7fce10104410 2023-12-14T18:37:02.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1010f230 con 0x7fce10104410 2023-12-14T18:37:02.235 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce0cff9700 1 -- 172.21.15.93:0/3922475011 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce08005440 con 0x7fce10104410 2023-12-14T18:37:02.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.229+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce1010f0a0 con 0x7fce10104410 2023-12-14T18:37:02.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.230+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1010f3c0 con 0x7fce10104410 2023-12-14T18:37:02.236 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.231+0000 7fce0cff9700 1 -- 172.21.15.93:0/3922475011 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fce08004ad0 con 0x7fce10104410 2023-12-14T18:37:02.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.231+0000 7fce0cff9700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcdfc0a12c0 0x7fcdfc0a3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:02.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.232+0000 7fce0f7fe700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcdfc0a12c0 0x7fcdfc0a3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:02.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.232+0000 7fce0cff9700 1 -- 172.21.15.93:0/3922475011 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 9536+0+0 (secure 0 0 0) 0x7fce080cc7f0 con 0x7fce10104410 2023-12-14T18:37:02.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.232+0000 7fce0f7fe700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcdfc0a12c0 0x7fcdfc0a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8000c00 tx=0x7fcdf8005960 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:02.237 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.236+0000 7fce0cff9700 1 -- 172.21.15.93:0/3922475011 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fce080958a0 con 0x7fce10104410 2023-12-14T18:37:02.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2956050415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:02.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/2692854174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:37:02.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/295239884' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-14T18:37:02.343 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-mon[43486]: osdmap e97: 12 total, 12 up, 12 in 2023-12-14T18:37:02.344 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-b[68560]: ignoring --setuser ceph since I am not root 2023-12-14T18:37:02.344 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-b[68560]: ignoring --setgroup ceph since I am not root 2023-12-14T18:37:02.344 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-b[68560]: starting mds.b at 2023-12-14T18:37:02.344 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-e[67879]: ignoring --setuser ceph since I am not root 2023-12-14T18:37:02.345 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-e[67879]: ignoring --setgroup ceph since I am not root 2023-12-14T18:37:02.345 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-e[67879]: starting mds.e at 2023-12-14T18:37:02.345 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-h[68220]: ignoring --setuser ceph since I am not root 2023-12-14T18:37:02.345 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-h[68220]: ignoring --setgroup ceph since I am not root 2023-12-14T18:37:02.345 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-mds[68243]: -- 172.21.15.97:0/2129883939 <== mon.0 v2:172.21.15.93:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55562ecf3520 con 0x55562ecc8c00 2023-12-14T18:37:02.345 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:37:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-h[68220]: starting mds.h at 2023-12-14T18:37:02.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:02.468+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fce101a5680 con 0x7fce10104410 2023-12-14T18:37:02.496 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-c[69302]: ignoring --setuser ceph since I am not root 2023-12-14T18:37:02.496 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-c[69302]: ignoring --setgroup ceph since I am not root 2023-12-14T18:37:02.496 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-mds[69325]: -- 172.21.15.160:0/3024265584 <== mon.1 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x5594acc7f520 con 0x5594acc55000 2023-12-14T18:37:02.496 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-c[69302]: starting mds.c at 2023-12-14T18:37:02.497 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-f[68962]: ignoring --setuser ceph since I am not root 2023-12-14T18:37:02.497 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-f[68962]: ignoring --setgroup ceph since I am not root 2023-12-14T18:37:02.497 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-mds[68985]: -- 172.21.15.160:0/2859292737 <== mon.2 v2:172.21.15.97:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x562ec83d3520 con 0x562ec83a8800 2023-12-14T18:37:02.497 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-f[68962]: starting mds.f at 2023-12-14T18:37:02.497 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-i[69643]: ignoring --setuser ceph since I am not root 2023-12-14T18:37:02.498 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-i[69643]: ignoring --setgroup ceph since I am not root 2023-12-14T18:37:02.498 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-mds[69666]: -- 172.21.15.160:0/3511622820 <== mon.1 v2:172.21.15.160:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55a20f869520 con 0x55a20f83e800 2023-12-14T18:37:02.498 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-i[69643]: starting mds.i at 2023-12-14T18:37:02.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2956050415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:02.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/2692854174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:37:02.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/295239884' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-14T18:37:02.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:02 smithi160 ceph-mon[43840]: osdmap e97: 12 total, 12 up, 12 in 2023-12-14T18:37:02.564 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:02 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2956050415' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2023-12-14T18:37:02.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:02 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/2692854174' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2023-12-14T18:37:02.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:02 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/295239884' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2023-12-14T18:37:02.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:02 smithi093 ceph-mon[42070]: osdmap e97: 12 total, 12 up, 12 in 2023-12-14T18:37:02.565 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-a[42047]: 2023-12-14T18:37:02.073+0000 7f3fa64d7700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-14T18:37:03.115 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.114+0000 7fce0cff9700 1 -- 172.21.15.93:0/3922475011 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v38) v1 ==== 104+0+0 (secure 0 0 0) 0x7fce08095070 con 0x7fce10104410 2023-12-14T18:37:03.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.124+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcdfc0a12c0 msgr2=0x7fcdfc0a3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:03.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.124+0000 7fce1601b700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcdfc0a12c0 0x7fcdfc0a3770 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8000c00 tx=0x7fcdf8005960 comp rx=0 tx=0).stop 2023-12-14T18:37:03.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.124+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce10104410 msgr2=0x7fce10195b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:03.125 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.124+0000 7fce1601b700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce10104410 0x7fce10195b00 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7fce0800ada0 tx=0x7fce0800c330 comp rx=0 tx=0).stop 2023-12-14T18:37:03.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.127+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 shutdown_connections 2023-12-14T18:37:03.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.127+0000 7fce1601b700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fcdfc0a12c0 0x7fcdfc0a3770 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcdf8000c00 tx=0x7fcdf8005960 comp rx=0 tx=0).stop 2023-12-14T18:37:03.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.127+0000 7fce1601b700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fce10104410 0x7fce10195b00 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:03.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.127+0000 7fce1601b700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fce100ff3d0 0x7fce10195080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:03.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.127+0000 7fce1601b700 1 --2- 172.21.15.93:0/3922475011 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fce100ffd70 0x7fce101955c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:03.128 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.127+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 >> 172.21.15.93:0/3922475011 conn(0x7fce10076000 msgr2=0x7fce10102d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:03.129 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.128+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 shutdown_connections 2023-12-14T18:37:03.133 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.133+0000 7fce1601b700 1 -- 172.21.15.93:0/3922475011 wait complete. 2023-12-14T18:37:03.161 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-14T18:37:03.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.407+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/546737107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410ae90 msgr2=0x7fbd9410b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:03.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.407+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/546737107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410ae90 0x7fbd9410b260 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fbd840099c0 tx=0x7fbd84009cd0 comp rx=0 tx=0).stop 2023-12-14T18:37:03.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.407+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/546737107 shutdown_connections 2023-12-14T18:37:03.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.407+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/546737107 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9410fed0 0x7fbd94117aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:03.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.407+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/546737107 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410ae90 0x7fbd9410b260 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:03.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.407+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/546737107 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd9410b830 0x7fbd9410f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:03.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.407+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/546737107 >> 172.21.15.93:0/546737107 conn(0x7fbd9406b1f0 msgr2=0x7fbd9406b5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:03.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.407+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/546737107 shutdown_connections 2023-12-14T18:37:03.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.407+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/546737107 wait complete. 2023-12-14T18:37:03.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.408+0000 7fbd9a3a6700 1 Processor -- start 2023-12-14T18:37:03.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.410+0000 7fbd9a3a6700 1 -- start start 2023-12-14T18:37:03.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.411+0000 7fbd9a3a6700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd9410ae90 0x7fbd941a0e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:03.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.411+0000 7fbd9a3a6700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410b830 0x7fbd941a1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:03.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.411+0000 7fbd9a3a6700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9410fed0 0x7fbd941a57e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:03.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.411+0000 7fbd9a3a6700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd941a5e20 con 0x7fbd9410ae90 2023-12-14T18:37:03.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.411+0000 7fbd9a3a6700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd941a5f90 con 0x7fbd9410fed0 2023-12-14T18:37:03.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.411+0000 7fbd9a3a6700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd941a6150 con 0x7fbd9410b830 2023-12-14T18:37:03.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.411+0000 7fbd93fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd9410ae90 0x7fbd941a0e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:03.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.411+0000 7fbd93fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd9410ae90 0x7fbd941a0e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45656/0 (socket says 172.21.15.93:45656) 2023-12-14T18:37:03.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.411+0000 7fbd93fff700 1 -- 172.21.15.93:0/290146599 learned_addr learned my addr 172.21.15.93:0/290146599 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:37:03.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.412+0000 7fbd93fff700 1 -- 172.21.15.93:0/290146599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410b830 msgr2=0x7fbd941a1380 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:37:03.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.412+0000 7fbd93fff700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410b830 0x7fbd941a1380 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:03.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.412+0000 7fbd93fff700 1 -- 172.21.15.93:0/290146599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9410fed0 msgr2=0x7fbd941a57e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-14T18:37:03.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.412+0000 7fbd93fff700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9410fed0 0x7fbd941a57e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:03.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.412+0000 7fbd93fff700 1 -- 172.21.15.93:0/290146599 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd840095f0 con 0x7fbd9410ae90 2023-12-14T18:37:03.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.412+0000 7fbd93fff700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd9410ae90 0x7fbd941a0e40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009990 tx=0x7fbd840047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:03.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.412+0000 7fbd917fa700 1 -- 172.21.15.93:0/290146599 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd8400fa80 con 0x7fbd9410ae90 2023-12-14T18:37:03.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.412+0000 7fbd917fa700 1 -- 172.21.15.93:0/290146599 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7fbd8402e070 con 0x7fbd9410ae90 2023-12-14T18:37:03.415 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.413+0000 7fbd917fa700 1 -- 172.21.15.93:0/290146599 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd840173f0 con 0x7fbd9410ae90 2023-12-14T18:37:03.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.413+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd9411adb0 con 0x7fbd9410ae90 2023-12-14T18:37:03.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.413+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd9411ac20 con 0x7fbd9410ae90 2023-12-14T18:37:03.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.415+0000 7fbd917fa700 1 -- 172.21.15.93:0/290146599 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7fbd84030070 con 0x7fbd9410ae90 2023-12-14T18:37:03.416 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.416+0000 7fbd917fa700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd7c0a14d0 0x7fbd7c0a3980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:03.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.416+0000 7fbd917fa700 1 -- 172.21.15.93:0/290146599 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 9712+0+0 (secure 0 0 0) 0x7fbd840dfb70 con 0x7fbd9410ae90 2023-12-14T18:37:03.417 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.416+0000 7fbd937fe700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd7c0a14d0 0x7fbd7c0a3980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:03.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.417+0000 7fbd937fe700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd7c0a14d0 0x7fbd7c0a3980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd941a2450 tx=0x7fbd88009260 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:03.418 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.417+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd9411af40 con 0x7fbd9410ae90 2023-12-14T18:37:03.424 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.423+0000 7fbd917fa700 1 -- 172.21.15.93:0/290146599 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7fbd840a8b30 con 0x7fbd9410ae90 2023-12-14T18:37:03.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:03 smithi160 ceph-mon[43840]: pgmap v330: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:03.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:03 smithi160 ceph-mon[43840]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-14T18:37:03.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:03 smithi160 ceph-mon[43840]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-14T18:37:03.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:03 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/295239884' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-14T18:37:03.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:03 smithi160 ceph-mon[43840]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-14T18:37:03.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:03 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3922475011' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:37:03.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:03 smithi160 ceph-mon[43840]: osdmap e98: 12 total, 12 up, 12 in 2023-12-14T18:37:03.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:03 smithi093 ceph-mon[42070]: pgmap v330: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:03.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:03 smithi093 ceph-mon[42070]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-14T18:37:03.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:03 smithi093 ceph-mon[42070]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-14T18:37:03.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:03 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/295239884' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-14T18:37:03.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:03 smithi093 ceph-mon[42070]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-14T18:37:03.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:03 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3922475011' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:37:03.566 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:03 smithi093 ceph-mon[42070]: osdmap e98: 12 total, 12 up, 12 in 2023-12-14T18:37:03.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:03 smithi097 ceph-mon[43486]: pgmap v330: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 KiB/s rd, 5 op/s 2023-12-14T18:37:03.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:03 smithi097 ceph-mon[43486]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2023-12-14T18:37:03.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:03 smithi097 ceph-mon[43486]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-14T18:37:03.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:03 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/295239884' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2023-12-14T18:37:03.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:03 smithi097 ceph-mon[43486]: fsmap cephfs:0/3 6 up:standby, 3 failed 2023-12-14T18:37:03.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:03 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3922475011' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2023-12-14T18:37:03.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:03 smithi097 ceph-mon[43486]: osdmap e98: 12 total, 12 up, 12 in 2023-12-14T18:37:03.764 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:03.759+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbd940679f0 con 0x7fbd9410ae90 2023-12-14T18:37:04.137 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.136+0000 7fbd917fa700 1 -- 172.21.15.93:0/290146599 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v99) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbd940679f0 con 0x7fbd9410ae90 2023-12-14T18:37:04.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd7c0a14d0 msgr2=0x7fbd7c0a3980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:04.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd7c0a14d0 0x7fbd7c0a3980 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd941a2450 tx=0x7fbd88009260 comp rx=0 tx=0).stop 2023-12-14T18:37:04.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd9410ae90 msgr2=0x7fbd941a0e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:04.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd9410ae90 0x7fbd941a0e40 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009990 tx=0x7fbd840047c0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 shutdown_connections 2023-12-14T18:37:04.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7fbd7c0a14d0 0x7fbd7c0a3980 secure :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fbd941a2450 tx=0x7fbd88009260 comp rx=0 tx=0).stop 2023-12-14T18:37:04.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7fbd9410ae90 0x7fbd941a0e40 secure :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7fbd84009990 tx=0x7fbd840047c0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7fbd9410fed0 0x7fbd941a57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.145 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 --2- 172.21.15.93:0/290146599 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7fbd9410b830 0x7fbd941a1380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.143+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 >> 172.21.15.93:0/290146599 conn(0x7fbd9406b1f0 msgr2=0x7fbd9410e7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:04.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.144+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 shutdown_connections 2023-12-14T18:37:04.146 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.146+0000 7fbd9a3a6700 1 -- 172.21.15.93:0/290146599 wait complete. 2023-12-14T18:37:04.153 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_metadata' removed 2023-12-14T18:37:04.170 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-14T18:37:04.408 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.401+0000 7f1ad979d700 1 -- 172.21.15.93:0/1334966585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad41072d0 msgr2=0x7f1ad4113590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:04.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.401+0000 7f1ad979d700 1 --2- 172.21.15.93:0/1334966585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad41072d0 0x7f1ad4113590 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f1ac401a9a0 tx=0x7f1ac401acb0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.406+0000 7f1ad979d700 1 -- 172.21.15.93:0/1334966585 shutdown_connections 2023-12-14T18:37:04.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.406+0000 7f1ad979d700 1 --2- 172.21.15.93:0/1334966585 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad41072d0 0x7f1ad4113590 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.406+0000 7f1ad979d700 1 --2- 172.21.15.93:0/1334966585 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad410c7f0 0x7f1ad410cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.406+0000 7f1ad979d700 1 --2- 172.21.15.93:0/1334966585 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad4106920 0x7f1ad4106d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.409 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.406+0000 7f1ad979d700 1 -- 172.21.15.93:0/1334966585 >> 172.21.15.93:0/1334966585 conn(0x7f1ad4075fc0 msgr2=0x7f1ad40783d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:04.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.407+0000 7f1ad979d700 1 -- 172.21.15.93:0/1334966585 shutdown_connections 2023-12-14T18:37:04.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.407+0000 7f1ad979d700 1 -- 172.21.15.93:0/1334966585 wait complete. 2023-12-14T18:37:04.410 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.408+0000 7f1ad979d700 1 Processor -- start 2023-12-14T18:37:04.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.408+0000 7f1ad979d700 1 -- start start 2023-12-14T18:37:04.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.408+0000 7f1ad979d700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad4106920 0x7f1ad410ed50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:04.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.408+0000 7f1ad979d700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad41072d0 0x7f1ad410f290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:04.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.408+0000 7f1ad979d700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad410c7f0 0x7f1ad41c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:04.411 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad979d700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad41c2380 con 0x7f1ad4106920 2023-12-14T18:37:04.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad979d700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad41c24c0 con 0x7f1ad41072d0 2023-12-14T18:37:04.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad979d700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ad41c2630 con 0x7f1ad410c7f0 2023-12-14T18:37:04.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad4106920 0x7f1ad410ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:04.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad3fff700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad4106920 0x7f1ad410ed50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.93:3300/0 says I am v2:172.21.15.93:45684/0 (socket says 172.21.15.93:45684) 2023-12-14T18:37:04.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad3fff700 1 -- 172.21.15.93:0/386637271 learned_addr learned my addr 172.21.15.93:0/386637271 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:37:04.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad3fff700 1 -- 172.21.15.93:0/386637271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad410c7f0 msgr2=0x7f1ad41c1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:04.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad3fff700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad410c7f0 0x7f1ad41c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.412 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad3fff700 1 -- 172.21.15.93:0/386637271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad41072d0 msgr2=0x7f1ad410f290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:04.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad3fff700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad41072d0 0x7f1ad410f290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:04.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.409+0000 7f1ad3fff700 1 -- 172.21.15.93:0/386637271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ac401a600 con 0x7f1ad4106920 2023-12-14T18:37:04.413 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.410+0000 7f1ad3fff700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad4106920 0x7f1ad410ed50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800b550 tx=0x7f1ac800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:04.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.411+0000 7f1ad17fa700 1 -- 172.21.15.93:0/386637271 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac8011830 con 0x7f1ad4106920 2023-12-14T18:37:04.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.411+0000 7f1ad17fa700 1 -- 172.21.15.93:0/386637271 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f1ac8011e50 con 0x7f1ad4106920 2023-12-14T18:37:04.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.411+0000 7f1ad17fa700 1 -- 172.21.15.93:0/386637271 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ac800f2e0 con 0x7f1ad4106920 2023-12-14T18:37:04.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.411+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ad407bb90 con 0x7f1ad4106920 2023-12-14T18:37:04.414 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.411+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ad407ba00 con 0x7f1ad4106920 2023-12-14T18:37:04.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.416+0000 7f1ad17fa700 1 -- 172.21.15.93:0/386637271 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f1ac800f440 con 0x7f1ad4106920 2023-12-14T18:37:04.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.417+0000 7f1ad17fa700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1abc0a1450 0x7f1abc0a3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:04.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.417+0000 7f1ad17fa700 1 -- 172.21.15.93:0/386637271 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(99..99 src has 1..99) v4 ==== 9099+0+0 (secure 0 0 0) 0x7f1ac80cc7a0 con 0x7f1ad4106920 2023-12-14T18:37:04.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.417+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ac0005320 con 0x7f1ad4106920 2023-12-14T18:37:04.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.418+0000 7f1ad37fe700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1abc0a1450 0x7f1abc0a3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:04.421 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.418+0000 7f1ad37fe700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1abc0a1450 0x7f1abc0a3900 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4110360 tx=0x7f1acc009250 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:04.430 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.427+0000 7f1ad17fa700 1 -- 172.21.15.93:0/386637271 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f1ac8095a80 con 0x7f1ad4106920 2023-12-14T18:37:04.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-14T18:37:04.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-14T18:37:04.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: Cluster is now healthy 2023-12-14T18:37:04.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.160:6834/1696681487,v1:172.21.15.160:6835/1696681487] up:boot 2023-12-14T18:37:04.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.97:6834/893996500,v1:172.21.15.97:6836/893996500] up:boot 2023-12-14T18:37:04.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.97:6838/2006575977,v1:172.21.15.97:6839/2006575977] up:boot 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.97:6835/3076247922,v1:172.21.15.97:6837/3076247922] up:boot 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.160:6836/552271530,v1:172.21.15.160:6837/552271530] up:boot 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: mds.? [v2:172.21.15.160:6838/2242604118,v1:172.21.15.160:6839/2242604118] up:boot 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/3922475011' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: fsmap 12 up:standby 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:37:04.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:37:04.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:37:04.499 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:04 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/290146599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:37:04.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-14T18:37:04.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-14T18:37:04.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: Cluster is now healthy 2023-12-14T18:37:04.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.160:6834/1696681487,v1:172.21.15.160:6835/1696681487] up:boot 2023-12-14T18:37:04.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.97:6834/893996500,v1:172.21.15.97:6836/893996500] up:boot 2023-12-14T18:37:04.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.97:6838/2006575977,v1:172.21.15.97:6839/2006575977] up:boot 2023-12-14T18:37:04.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.97:6835/3076247922,v1:172.21.15.97:6837/3076247922] up:boot 2023-12-14T18:37:04.567 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.160:6836/552271530,v1:172.21.15.160:6837/552271530] up:boot 2023-12-14T18:37:04.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: mds.? [v2:172.21.15.160:6838/2242604118,v1:172.21.15.160:6839/2242604118] up:boot 2023-12-14T18:37:04.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/3922475011' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:37:04.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: fsmap 12 up:standby 2023-12-14T18:37:04.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:37:04.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:37:04.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:37:04.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:37:04.568 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:37:04.569 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:37:04.569 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:04 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/290146599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:37:04.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2023-12-14T18:37:04.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2023-12-14T18:37:04.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: Cluster is now healthy 2023-12-14T18:37:04.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.160:6834/1696681487,v1:172.21.15.160:6835/1696681487] up:boot 2023-12-14T18:37:04.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.97:6834/893996500,v1:172.21.15.97:6836/893996500] up:boot 2023-12-14T18:37:04.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.97:6838/2006575977,v1:172.21.15.97:6839/2006575977] up:boot 2023-12-14T18:37:04.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.97:6835/3076247922,v1:172.21.15.97:6837/3076247922] up:boot 2023-12-14T18:37:04.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.160:6836/552271530,v1:172.21.15.160:6837/552271530] up:boot 2023-12-14T18:37:04.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: mds.? [v2:172.21.15.160:6838/2242604118,v1:172.21.15.160:6839/2242604118] up:boot 2023-12-14T18:37:04.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/3922475011' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2023-12-14T18:37:04.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: fsmap 12 up:standby 2023-12-14T18:37:04.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2023-12-14T18:37:04.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2023-12-14T18:37:04.595 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2023-12-14T18:37:04.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2023-12-14T18:37:04.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2023-12-14T18:37:04.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: from='mgr.25100 172.21.15.93:0/2625805918' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2023-12-14T18:37:04.596 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:04 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/290146599' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:37:04.770 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:04.769+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f1ac0005190 con 0x7f1ad4106920 2023-12-14T18:37:05.165 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.164+0000 7f1ad17fa700 1 -- 172.21.15.93:0/386637271 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v100) v1 ==== 169+0+0 (secure 0 0 0) 0x7f1ac8015030 con 0x7f1ad4106920 2023-12-14T18:37:05.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1abc0a1450 msgr2=0x7f1abc0a3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:05.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1abc0a1450 0x7f1abc0a3900 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1ad4110360 tx=0x7f1acc009250 comp rx=0 tx=0).stop 2023-12-14T18:37:05.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad4106920 msgr2=0x7f1ad410ed50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:05.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad4106920 0x7f1ad410ed50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800b550 tx=0x7f1ac800b910 comp rx=0 tx=0).stop 2023-12-14T18:37:05.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 shutdown_connections 2023-12-14T18:37:05.182 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f1abc0a1450 0x7f1abc0a3900 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:05.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f1ad4106920 0x7f1ad410ed50 secure :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f1ac800b550 tx=0x7f1ac800b910 comp rx=0 tx=0).stop 2023-12-14T18:37:05.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f1ad410c7f0 0x7f1ad41c1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:05.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 --2- 172.21.15.93:0/386637271 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f1ad41072d0 0x7f1ad410f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:05.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.179+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 >> 172.21.15.93:0/386637271 conn(0x7f1ad4075fc0 msgr2=0x7f1ad410a2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:05.183 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.180+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 shutdown_connections 2023-12-14T18:37:05.185 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.184+0000 7f1ad979d700 1 -- 172.21.15.93:0/386637271 wait complete. 2023-12-14T18:37:05.206 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data' removed 2023-12-14T18:37:05.227 DEBUG:teuthology.orchestra.run.smithi093:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2023-12-14T18:37:05.461 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 -- 172.21.15.93:0/1642658755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7280100080 msgr2=0x7f7280104640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:05.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 --2- 172.21.15.93:0/1642658755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7280100080 0x7f7280104640 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f727400aad0 tx=0x7f727400ade0 comp rx=0 tx=0).stop 2023-12-14T18:37:05.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 -- 172.21.15.93:0/1642658755 shutdown_connections 2023-12-14T18:37:05.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 --2- 172.21.15.93:0/1642658755 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f7280104d70 0x7f728010b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:05.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 --2- 172.21.15.93:0/1642658755 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f72800ff6e0 0x7f72800ffab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:05.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 --2- 172.21.15.93:0/1642658755 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7280100080 0x7f7280104640 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:05.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 -- 172.21.15.93:0/1642658755 >> 172.21.15.93:0/1642658755 conn(0x7f7280074a60 msgr2=0x7f7280074e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:05.462 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 -- 172.21.15.93:0/1642658755 shutdown_connections 2023-12-14T18:37:05.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 -- 172.21.15.93:0/1642658755 wait complete. 2023-12-14T18:37:05.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 Processor -- start 2023-12-14T18:37:05.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.460+0000 7f72863aa700 1 -- start start 2023-12-14T18:37:05.463 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f72863aa700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72800ff6e0 0x7f7280194360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:05.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f72863aa700 1 --2- >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7280100080 0x7f72801948a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:05.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f72863aa700 1 --2- >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7280104d70 0x7f7280198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:05.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f72863aa700 1 -- --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72801992f0 con 0x7f7280100080 2023-12-14T18:37:05.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f72863aa700 1 -- --> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280199460 con 0x7f72800ff6e0 2023-12-14T18:37:05.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f72863aa700 1 -- --> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7280199620 con 0x7f7280104d70 2023-12-14T18:37:05.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f727ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72800ff6e0 0x7f7280194360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:05.464 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f727ffff700 1 --2- >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72800ff6e0 0x7f7280194360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.97:3300/0 says I am v2:172.21.15.93:47236/0 (socket says 172.21.15.93:47236) 2023-12-14T18:37:05.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f727ffff700 1 -- 172.21.15.93:0/1109408539 learned_addr learned my addr 172.21.15.93:0/1109408539 (peer_addr_for_me v2:172.21.15.93:0/0) 2023-12-14T18:37:05.465 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.461+0000 7f727f7fe700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7280100080 0x7f72801948a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:05.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.462+0000 7f727f7fe700 1 -- 172.21.15.93:0/1109408539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7280104d70 msgr2=0x7f7280198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:05.468 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.462+0000 7f727f7fe700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7280104d70 0x7f7280198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:05.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.462+0000 7f727f7fe700 1 -- 172.21.15.93:0/1109408539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72800ff6e0 msgr2=0x7f7280194360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:05.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.462+0000 7f727f7fe700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72800ff6e0 0x7f7280194360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:05.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.462+0000 7f727f7fe700 1 -- 172.21.15.93:0/1109408539 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727400a700 con 0x7f7280100080 2023-12-14T18:37:05.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.462+0000 7f727f7fe700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7280100080 0x7f72801948a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7280100450 tx=0x7f7274004340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:05.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.462+0000 7f727d7fa700 1 -- 172.21.15.93:0/1109408539 <== mon.0 v2:172.21.15.93:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274003ac0 con 0x7f7280100080 2023-12-14T18:37:05.469 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.462+0000 7f727d7fa700 1 -- 172.21.15.93:0/1109408539 <== mon.0 v2:172.21.15.93:3300/0 2 ==== config(26 keys) v1 ==== 1105+0+0 (secure 0 0 0) 0x7f727400e910 con 0x7f7280100080 2023-12-14T18:37:05.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.462+0000 7f727d7fa700 1 -- 172.21.15.93:0/1109408539 <== mon.0 v2:172.21.15.93:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274016910 con 0x7f7280100080 2023-12-14T18:37:05.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.463+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f728010e1b0 con 0x7f7280100080 2023-12-14T18:37:05.470 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.463+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f728010e020 con 0x7f7280100080 2023-12-14T18:37:05.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.468+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f728010e340 con 0x7f7280100080 2023-12-14T18:37:05.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.468+0000 7f727d7fa700 1 -- 172.21.15.93:0/1109408539 <== mon.0 v2:172.21.15.93:3300/0 4 ==== mgrmap(e 25) v1 ==== 135846+0+0 (secure 0 0 0) 0x7f727400e450 con 0x7f7280100080 2023-12-14T18:37:05.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.469+0000 7f727d7fa700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f72680a1340 0x7f72680a37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-14T18:37:05.472 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.469+0000 7f727d7fa700 1 -- 172.21.15.93:0/1109408539 <== mon.0 v2:172.21.15.93:3300/0 5 ==== osd_map(100..100 src has 1..100) v4 ==== 8598+0+0 (secure 0 0 0) 0x7f7274012070 con 0x7f7280100080 2023-12-14T18:37:05.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.477+0000 7f727ffff700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f72680a1340 0x7f72680a37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-14T18:37:05.478 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.477+0000 7f727d7fa700 1 -- 172.21.15.93:0/1109408539 <== mon.0 v2:172.21.15.93:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+177948 (secure 0 0 0) 0x7f7274098f90 con 0x7f7280100080 2023-12-14T18:37:05.480 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.479+0000 7f727ffff700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f72680a1340 0x7f72680a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72700098f0 tx=0x7f7270008040 comp rx=0 tx=0).ready entity=mgr.25100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-14T18:37:05.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:05 smithi160 ceph-mon[43840]: pgmap v332: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-12-14T18:37:05.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:05 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/290146599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:37:05.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:05 smithi160 ceph-mon[43840]: osdmap e99: 12 total, 12 up, 12 in 2023-12-14T18:37:05.498 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:05 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/386637271' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:37:05.569 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:05 smithi093 ceph-mon[42070]: pgmap v332: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-12-14T18:37:05.570 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:05 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/290146599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:37:05.570 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:05 smithi093 ceph-mon[42070]: osdmap e99: 12 total, 12 up, 12 in 2023-12-14T18:37:05.570 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:05 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/386637271' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:37:05.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:05 smithi097 ceph-mon[43486]: pgmap v332: 129 pgs: 129 active+clean; 732 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 3 op/s 2023-12-14T18:37:05.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:05 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/290146599' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:37:05.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:05 smithi097 ceph-mon[43486]: osdmap e99: 12 total, 12 up, 12 in 2023-12-14T18:37:05.594 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:05 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/386637271' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:37:05.807 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:05.806+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 --> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f7280106e10 con 0x7f7280100080 2023-12-14T18:37:06.184 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.182+0000 7f727d7fa700 1 -- 172.21.15.93:0/1109408539 <== mon.0 v2:172.21.15.93:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v101) v1 ==== 178+0+0 (secure 0 0 0) 0x7f727409c5b0 con 0x7f7280100080 2023-12-14T18:37:06.191 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.190+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f72680a1340 msgr2=0x7f72680a37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:06.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.190+0000 7f72863aa700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f72680a1340 0x7f72680a37f0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72700098f0 tx=0x7f7270008040 comp rx=0 tx=0).stop 2023-12-14T18:37:06.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.191+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7280100080 msgr2=0x7f72801948a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-14T18:37:06.192 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.191+0000 7f72863aa700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7280100080 0x7f72801948a0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7280100450 tx=0x7f7274004340 comp rx=0 tx=0).stop 2023-12-14T18:37:06.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.191+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 shutdown_connections 2023-12-14T18:37:06.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.191+0000 7f72863aa700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.160:3300/0,v1:172.21.15.160:6789/0] conn(0x7f7280104d70 0x7f7280198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:06.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.191+0000 7f72863aa700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.97:3300/0,v1:172.21.15.97:6789/0] conn(0x7f72800ff6e0 0x7f7280194360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-14T18:37:06.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.191+0000 7f72863aa700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:6800/3274300600,v1:172.21.15.93:6801/3274300600] conn(0x7f72680a1340 0x7f72680a37f0 secure :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f72700098f0 tx=0x7f7270008040 comp rx=0 tx=0).stop 2023-12-14T18:37:06.193 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.191+0000 7f72863aa700 1 --2- 172.21.15.93:0/1109408539 >> [v2:172.21.15.93:3300/0,v1:172.21.15.93:6789/0] conn(0x7f7280100080 0x7f72801948a0 secure :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f7280100450 tx=0x7f7274004340 comp rx=0 tx=0).stop 2023-12-14T18:37:06.194 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.191+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 >> 172.21.15.93:0/1109408539 conn(0x7f7280074a60 msgr2=0x7f72800fec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-14T18:37:06.203 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.201+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 shutdown_connections 2023-12-14T18:37:06.209 INFO:teuthology.orchestra.run.smithi093.stderr:2023-12-14T18:37:06.205+0000 7f72863aa700 1 -- 172.21.15.93:0/1109408539 wait complete. 2023-12-14T18:37:06.227 INFO:teuthology.orchestra.run.smithi093.stderr:pool 'cephfs_data_ec' removed 2023-12-14T18:37:06.250 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:37:06.301 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:37:06.338 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:37:06.370 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2023-12-14T18:37:06.370 DEBUG:teuthology.orchestra.run.smithi093:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-14T18:37:06.388 DEBUG:teuthology.orchestra.run.smithi097:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-14T18:37:06.405 DEBUG:teuthology.orchestra.run.smithi160:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2023-12-14T18:37:06.427 INFO:tasks.cephadm:Stopping all daemons... 2023-12-14T18:37:06.428 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2023-12-14T18:37:06.428 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.a 2023-12-14T18:37:06.451 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:06 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/386637271' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:37:06.451 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:06 smithi093 ceph-mon[42070]: osdmap e100: 12 total, 12 up, 12 in 2023-12-14T18:37:06.452 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:06 smithi093 ceph-mon[42070]: from='client.? 172.21.15.93:0/1109408539' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:37:06.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:06 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/386637271' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:37:06.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:06 smithi160 ceph-mon[43840]: osdmap e100: 12 total, 12 up, 12 in 2023-12-14T18:37:06.497 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:06 smithi160 ceph-mon[43840]: from='client.? 172.21.15.93:0/1109408539' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:37:06.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:06 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/386637271' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2023-12-14T18:37:06.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:06 smithi097 ceph-mon[43486]: osdmap e100: 12 total, 12 up, 12 in 2023-12-14T18:37:06.593 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:06 smithi097 ceph-mon[43486]: from='client.? 172.21.15.93:0/1109408539' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2023-12-14T18:37:06.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:06 smithi093 systemd[1]: Stopping Ceph mon.a for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:06.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:06 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-a[42047]: 2023-12-14T18:37:06.686+0000 7f3fa94dd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:06.816 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:06 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-a[42047]: 2023-12-14T18:37:06.686+0000 7f3fa94dd700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2023-12-14T18:37:07.315 INFO:journalctl@ceph.mon.a.smithi093.stdout:Dec 14 18:37:07 smithi093 bash[201504]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-a 2023-12-14T18:37:07.470 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.a.service' 2023-12-14T18:37:07.574 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:07.574 INFO:tasks.cephadm.mon.a:Stopped mon.a 2023-12-14T18:37:07.575 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2023-12-14T18:37:07.575 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.b 2023-12-14T18:37:08.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:07 smithi097 systemd[1]: Stopping Ceph mon.b for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:08.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:07 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-b[43463]: 2023-12-14T18:37:07.834+0000 7fa22d118700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:08.093 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:07 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-b[43463]: 2023-12-14T18:37:07.834+0000 7fa22d118700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2023-12-14T18:37:08.566 INFO:journalctl@ceph.mon.b.smithi097.stdout:Dec 14 18:37:08 smithi097 bash[74418]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-b 2023-12-14T18:37:08.573 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.b.service' 2023-12-14T18:37:08.624 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:08.625 INFO:tasks.cephadm.mon.c:Stopped mon.b 2023-12-14T18:37:08.625 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2023-12-14T18:37:08.625 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.c 2023-12-14T18:37:08.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:08 smithi160 systemd[1]: Stopping Ceph mon.c for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:08.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-c[43802]: 2023-12-14T18:37:08.861+0000 7f265fabc700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:08.996 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-c[43802]: 2023-12-14T18:37:08.861+0000 7f265fabc700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2023-12-14T18:37:09.496 INFO:journalctl@ceph.mon.c.smithi160.stdout:Dec 14 18:37:09 smithi160 bash[75644]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mon-c 2023-12-14T18:37:09.547 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mon.c.service' 2023-12-14T18:37:09.603 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:09.603 INFO:tasks.cephadm.mon.c:Stopped mon.c 2023-12-14T18:37:09.603 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2023-12-14T18:37:09.604 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.x 2023-12-14T18:37:09.950 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:37:09 smithi093 systemd[1]: Stopping Ceph mgr.x for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:09.950 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:37:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:37:09.827+0000 7fc630cde700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:09.951 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:37:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x[42324]: 2023-12-14T18:37:09.827+0000 7fc630cde700 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2023-12-14T18:37:10.314 INFO:journalctl@ceph.mgr.x.smithi093.stdout:Dec 14 18:37:09 smithi093 bash[201645]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-x 2023-12-14T18:37:10.344 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.x.service' 2023-12-14T18:37:10.384 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:10.385 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2023-12-14T18:37:10.385 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2023-12-14T18:37:10.385 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.y 2023-12-14T18:37:10.740 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:37:10 smithi097 systemd[1]: Stopping Ceph mgr.y for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:11.093 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:37:10 smithi097 bash[74564]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-y 2023-12-14T18:37:11.093 INFO:journalctl@ceph.mgr.y.smithi097.stdout:Dec 14 18:37:10 smithi097 systemd[1]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.y.service: Main process exited, code=exited, status=143/n/a 2023-12-14T18:37:11.108 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.y.service' 2023-12-14T18:37:11.145 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:11.145 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2023-12-14T18:37:11.145 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2023-12-14T18:37:11.145 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.z 2023-12-14T18:37:11.496 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:37:11 smithi160 systemd[1]: Stopping Ceph mgr.z for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:11.905 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:37:11 smithi160 bash[75783]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mgr-z 2023-12-14T18:37:11.906 INFO:journalctl@ceph.mgr.z.smithi160.stdout:Dec 14 18:37:11 smithi160 systemd[1]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.z.service: Main process exited, code=exited, status=143/n/a 2023-12-14T18:37:11.912 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mgr.z.service' 2023-12-14T18:37:11.950 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:11.950 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2023-12-14T18:37:11.951 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2023-12-14T18:37:11.951 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.0 2023-12-14T18:37:12.315 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:37:11 smithi093 systemd[1]: Stopping Ceph osd.0 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:12.315 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:37:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-0[54878]: 2023-12-14T18:37:12.168+0000 7f08b7e57700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:12.315 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:37:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-0[54878]: 2023-12-14T18:37:12.168+0000 7f08b7e57700 -1 osd.0 101 *** Got signal Terminated *** 2023-12-14T18:37:12.316 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:37:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-0[54878]: 2023-12-14T18:37:12.168+0000 7f08b7e57700 -1 osd.0 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:37:18.315 INFO:journalctl@ceph.osd.0.smithi093.stdout:Dec 14 18:37:17 smithi093 bash[201789]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-0 2023-12-14T18:37:20.307 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.0.service' 2023-12-14T18:37:20.350 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:20.350 INFO:tasks.cephadm.osd.0:Stopped osd.0 2023-12-14T18:37:20.350 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2023-12-14T18:37:20.351 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.1 2023-12-14T18:37:20.843 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:37:20 smithi097 systemd[1]: Stopping Ceph osd.1 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:20.843 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:37:20 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-1[48411]: 2023-12-14T18:37:20.599+0000 7fb23ee86700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:20.843 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:37:20 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-1[48411]: 2023-12-14T18:37:20.599+0000 7fb23ee86700 -1 osd.1 101 *** Got signal Terminated *** 2023-12-14T18:37:20.843 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:37:20 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-1[48411]: 2023-12-14T18:37:20.599+0000 7fb23ee86700 -1 osd.1 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:37:26.593 INFO:journalctl@ceph.osd.1.smithi097.stdout:Dec 14 18:37:26 smithi097 bash[74708]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-1 2023-12-14T18:37:27.437 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.1.service' 2023-12-14T18:37:27.478 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:27.478 INFO:tasks.cephadm.osd.1:Stopped osd.1 2023-12-14T18:37:27.478 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2023-12-14T18:37:27.479 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.2 2023-12-14T18:37:27.996 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:37:27 smithi160 systemd[1]: Stopping Ceph osd.2 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:27.996 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:37:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-2[49563]: 2023-12-14T18:37:27.729+0000 7f1ee2be9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:27.996 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:37:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-2[49563]: 2023-12-14T18:37:27.729+0000 7f1ee2be9700 -1 osd.2 101 *** Got signal Terminated *** 2023-12-14T18:37:27.996 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:37:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-2[49563]: 2023-12-14T18:37:27.729+0000 7f1ee2be9700 -1 osd.2 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:37:33.746 INFO:journalctl@ceph.osd.2.smithi160.stdout:Dec 14 18:37:33 smithi160 bash[75929]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-2 2023-12-14T18:37:34.580 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.2.service' 2023-12-14T18:37:34.620 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:34.621 INFO:tasks.cephadm.osd.2:Stopped osd.2 2023-12-14T18:37:34.621 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2023-12-14T18:37:34.621 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.3 2023-12-14T18:37:35.065 INFO:journalctl@ceph.osd.3.smithi093.stdout:Dec 14 18:37:34 smithi093 systemd[1]: Stopping Ceph osd.3 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:35.065 INFO:journalctl@ceph.osd.3.smithi093.stdout:Dec 14 18:37:34 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-3[60361]: 2023-12-14T18:37:34.873+0000 7f097c0cb700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:35.065 INFO:journalctl@ceph.osd.3.smithi093.stdout:Dec 14 18:37:34 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-3[60361]: 2023-12-14T18:37:34.873+0000 7f097c0cb700 -1 osd.3 101 *** Got signal Terminated *** 2023-12-14T18:37:35.065 INFO:journalctl@ceph.osd.3.smithi093.stdout:Dec 14 18:37:34 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-3[60361]: 2023-12-14T18:37:34.873+0000 7f097c0cb700 -1 osd.3 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:37:39.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:39 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:39.148+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:39.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:39 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:39.204+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:39.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:39 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:39.215+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:39.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:39 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:39.655+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:40.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:39 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:39.831+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:40.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:40 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:40.246+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:40.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:40 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:40.170+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:40.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:40 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:40.212+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:40.815 INFO:journalctl@ceph.osd.3.smithi093.stdout:Dec 14 18:37:40 smithi093 bash[202047]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-3 2023-12-14T18:37:40.815 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:40 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:40.678+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:40.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:40 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:40.656+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:41.085 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:40 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:40.828+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:41.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:41 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:41.289+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:41.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:41 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:41.166+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:41.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:41 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:41.233+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:41.778 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.3.service' 2023-12-14T18:37:41.818 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:41.818 INFO:tasks.cephadm.osd.3:Stopped osd.3 2023-12-14T18:37:41.819 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2023-12-14T18:37:41.819 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.4 2023-12-14T18:37:41.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:41 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:41.683+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:42.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:41 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:41.713+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:42.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:41 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:41.787+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:42.195 INFO:journalctl@ceph.osd.4.smithi097.stdout:Dec 14 18:37:41 smithi097 systemd[1]: Stopping Ceph osd.4 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:42.195 INFO:journalctl@ceph.osd.4.smithi097.stdout:Dec 14 18:37:42 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-4[53874]: 2023-12-14T18:37:42.071+0000 7fcad05ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:42.195 INFO:journalctl@ceph.osd.4.smithi097.stdout:Dec 14 18:37:42 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-4[53874]: 2023-12-14T18:37:42.071+0000 7fcad05ab700 -1 osd.4 101 *** Got signal Terminated *** 2023-12-14T18:37:42.195 INFO:journalctl@ceph.osd.4.smithi097.stdout:Dec 14 18:37:42 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-4[53874]: 2023-12-14T18:37:42.071+0000 7fcad05ab700 -1 osd.4 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:37:42.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:42 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:42.275+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:42.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:42 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:42.125+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:42.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:42 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:42.193+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:42.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:42 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:42.715+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:43.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:42 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:42.752+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:43.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:42 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:42.744+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:43.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:43 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:43.299+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:43.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:43 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:43.113+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:43.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:43 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:43.172+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:43.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:43 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:43.743+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:44.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:43 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:43.695+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:44.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:43 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:43.799+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:44.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:44 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:44.257+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:44.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:44 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:44.066+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:44.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:44 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:44.138+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:44.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:44 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:44.708+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:45.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:44 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:44.806+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:45.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:44 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:44.684+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:45.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:45 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:45.240+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:45.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:45 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:45.049+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:45.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:45 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:45.101+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:45.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:45 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:45.699+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:46.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:45 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:45.769+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:46.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:45 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:45.733+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:46.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:46 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:46.061+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:46.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:46 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:46.260+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:46.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:46 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:46.052+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:46.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:46 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:46.722+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:47.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:46 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:46.738+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:47.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:46 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:46.779+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:47.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:46 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:46.779+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:47.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:47 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:47.068+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:47.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:47 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:47.064+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:47.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:47 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:47.064+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:47.497 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:47 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:47.228+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:47.497 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:47 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:47.228+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:48.023 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:47 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:47.745+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:48.023 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:47 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:47.745+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.395112+0000 front 2023-12-14T18:37:23.394954+0000 (oldest deadline 2023-12-14T18:37:47.395533+0000) 2023-12-14T18:37:48.064 INFO:journalctl@ceph.osd.4.smithi097.stdout:Dec 14 18:37:47 smithi097 bash[74962]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-4 2023-12-14T18:37:48.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:47 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:47.777+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:48.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:47 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:47.809+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:48.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:47 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:47.809+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:48.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:48 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:48.061+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:48.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:48 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:48.061+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:48.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:48 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:48.253+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:48.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:48 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:48.253+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:48.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:48 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:48.022+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:48.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:48 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:48.022+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:49.009 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.4.service' 2023-12-14T18:37:49.050 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:49.050 INFO:tasks.cephadm.osd.4:Stopped osd.4 2023-12-14T18:37:49.050 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2023-12-14T18:37:49.050 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.5 2023-12-14T18:37:49.055 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:48 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:48.790+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:49.055 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:48 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:48.790+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.395112+0000 front 2023-12-14T18:37:23.394954+0000 (oldest deadline 2023-12-14T18:37:47.395533+0000) 2023-12-14T18:37:49.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:48 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:48.817+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:49.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:48 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:48.793+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:49.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:48 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:48.793+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:49.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:49 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:49.047+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:49.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:49 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:49.047+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:49.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:49 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:49.244+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:49.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:49 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:49.244+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:49.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:49 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:49.053+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:49.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:49 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:49.053+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:49.497 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:49 smithi160 systemd[1]: Stopping Ceph osd.5 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:49.497 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:49 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:49.289+0000 7f9583b8f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:49.497 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:49 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:49.289+0000 7f9583b8f700 -1 osd.5 101 *** Got signal Terminated *** 2023-12-14T18:37:49.497 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:49 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:49.289+0000 7f9583b8f700 -1 osd.5 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:37:49.842 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:49 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:49.565+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:50.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:49 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:49.802+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:50.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:49 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:49.802+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:50.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:49 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:49.807+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:50.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:49 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:49.807+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:50.076 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:49 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:49.768+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:50.076 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:49 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:49.768+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.395112+0000 front 2023-12-14T18:37:23.394954+0000 (oldest deadline 2023-12-14T18:37:47.395533+0000) 2023-12-14T18:37:50.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:50 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:50.009+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:50.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:50 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:50.009+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:50.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:50 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:50.270+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:50.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:50 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:50.270+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:50.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:50 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:50.075+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:50.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:50 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:50.075+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:50.979 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:50 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:50.612+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:51.036 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:50 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:50.748+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:51.087 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:50 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:50.748+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.395112+0000 front 2023-12-14T18:37:23.394954+0000 (oldest deadline 2023-12-14T18:37:47.395533+0000) 2023-12-14T18:37:51.314 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:50 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:50.849+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:51.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:50 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:50.849+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:51.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:50 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:50.852+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:51.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:50 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:50.852+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:51.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:50 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:50.978+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:51.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:50 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:50.978+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:51.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:51 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:51.222+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:51.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:51 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:51.222+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:51.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:51 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:51.035+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:51.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:51 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:51.035+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:51.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:51 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:51.715+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:51.996 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:51 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:51.715+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.395112+0000 front 2023-12-14T18:37:23.394954+0000 (oldest deadline 2023-12-14T18:37:47.395533+0000) 2023-12-14T18:37:51.999 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:51 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:51.655+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:52.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:51 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:51.846+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:52.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:51 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:51.846+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:52.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:51 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:51.848+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:52.316 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:51 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:51.848+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:52.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:51 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:51.998+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:52.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:51 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:51.998+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:52.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:52 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:52.190+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:52.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:52 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:52.190+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:52.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:52 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:52.010+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:52.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:52 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:52.010+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:52.967 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:52 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:52.702+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:52.967 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:52 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:52.702+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.395112+0000 front 2023-12-14T18:37:23.394954+0000 (oldest deadline 2023-12-14T18:37:47.395533+0000) 2023-12-14T18:37:52.978 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:52 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:52.630+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:53.246 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:53 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:53.156+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:53.246 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:53 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:53.156+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:53.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:52 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:52.965+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:53.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:52 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:52.965+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:53.314 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:52 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:52.842+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:53.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:52 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:52.842+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:53.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:52 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:52.817+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:53.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:52 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:52.817+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:53.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:52 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:52.977+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:53.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:52 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:52.977+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:53.987 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:53 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:53.634+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:54.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:53.810+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:54.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:53.810+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:54.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:53.810+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:29.212344+0000 front 2023-12-14T18:37:29.212484+0000 (oldest deadline 2023-12-14T18:37:53.312304+0000) 2023-12-14T18:37:54.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:53.842+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:54.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:53.842+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:54.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:53 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:53.842+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:37:54.139 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:53 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:53.926+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:54.140 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:53 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:53.926+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:54.140 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:53 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:53.750+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.593787+0000 front 2023-12-14T18:37:14.593772+0000 (oldest deadline 2023-12-14T18:37:39.293851+0000) 2023-12-14T18:37:54.140 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:53 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:53.750+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.395112+0000 front 2023-12-14T18:37:23.394954+0000 (oldest deadline 2023-12-14T18:37:47.395533+0000) 2023-12-14T18:37:54.140 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:53 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5[54841]: 2023-12-14T18:37:53.750+0000 7f957e181700 -1 osd.5 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:27.395886+0000 front 2023-12-14T18:37:27.395904+0000 (oldest deadline 2023-12-14T18:37:53.296013+0000) 2023-12-14T18:37:54.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:53 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:53.986+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:54.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:53 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:53.986+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:54.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:54 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:54.139+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:54.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:54 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:54.139+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:55.023 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:54 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:54.629+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:55.246 INFO:journalctl@ceph.osd.5.smithi160.stdout:Dec 14 18:37:54 smithi160 bash[76181]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-5 2023-12-14T18:37:55.246 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:55 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:55.103+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:55.246 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:55 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:55.103+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:55.247 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:54 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:54.928+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:55.247 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:54 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:54.928+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:55.314 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:54.820+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:55.314 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:54.820+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:55.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:54.820+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:29.212344+0000 front 2023-12-14T18:37:29.212484+0000 (oldest deadline 2023-12-14T18:37:53.312304+0000) 2023-12-14T18:37:55.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:54.839+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:55.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:54.839+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:55.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:54 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:54.839+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:37:55.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:55 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:55.022+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:55.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:55 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:55.022+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:55.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:55 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:55.022+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:37:56.006 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:55 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:55.592+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:56.118 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:55 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:55.879+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:56.118 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:55 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:55.880+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:56.285 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.5.service' 2023-12-14T18:37:56.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:55 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:55.829+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:56.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:55 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:55.829+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:56.315 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:55 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:55.829+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:29.212344+0000 front 2023-12-14T18:37:29.212484+0000 (oldest deadline 2023-12-14T18:37:53.312304+0000) 2023-12-14T18:37:56.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:55 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:55.836+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:56.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:55 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:55.836+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:56.316 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:55 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:55.836+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:37:56.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:56 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:56.004+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:56.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:56 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:56.004+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:56.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:56 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:56.004+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:37:56.366 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:37:56.366 INFO:tasks.cephadm.osd.5:Stopped osd.5 2023-12-14T18:37:56.366 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2023-12-14T18:37:56.366 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.6 2023-12-14T18:37:56.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:56 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:56.152+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:56.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:56 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:56.152+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:56.655 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:56 smithi093 systemd[1]: Stopping Ceph osd.6 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:37:57.046 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:56 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:56.626+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:57.046 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:56 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:56.626+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:37:57.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:56.846+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:57.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:56.846+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:57.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:56.846+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:37:57.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:56.654+0000 7fa7d229b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:37:57.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:56.654+0000 7fa7d229b700 -1 osd.6 101 *** Got signal Terminated *** 2023-12-14T18:37:57.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:56.654+0000 7fa7d229b700 -1 osd.6 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:37:57.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:56.793+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:57.066 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:56.793+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:57.066 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:56 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:56.793+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:29.212344+0000 front 2023-12-14T18:37:29.212484+0000 (oldest deadline 2023-12-14T18:37:53.312304+0000) 2023-12-14T18:37:57.172 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:56 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:56.833+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:57.173 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:56 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:56.833+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:57.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:57 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:57.045+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:57.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:57 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:57.045+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:57.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:57 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:57.045+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:37:57.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:57 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:57.172+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:57.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:57 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:57.172+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:58.025 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:57 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:57.617+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:58.026 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:57 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:57.617+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:37:58.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:57 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:57.767+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:58.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:57 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:57.767+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:58.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:57 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:57.767+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:29.212344+0000 front 2023-12-14T18:37:29.212484+0000 (oldest deadline 2023-12-14T18:37:53.312304+0000) 2023-12-14T18:37:58.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:57 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:57.801+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:58.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:57 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:57.801+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:58.066 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:57 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:57.801+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:37:58.172 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:57 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:57.872+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:58.172 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:57 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:57.872+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:58.172 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:57 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:57.872+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:37:58.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:58 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:58.024+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:58.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:58 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:58.024+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:58.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:58 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:58.024+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:37:58.495 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:58 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:58.170+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:58.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:58 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:58.170+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:59.056 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:58 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:58.635+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:37:59.057 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:58 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:58.635+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:37:59.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:58 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:58.784+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:37:59.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:58 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:58.784+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:37:59.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:58 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:58.784+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:29.212344+0000 front 2023-12-14T18:37:29.212484+0000 (oldest deadline 2023-12-14T18:37:53.312304+0000) 2023-12-14T18:37:59.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:58 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:58.806+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:37:59.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:58 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:58.806+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:37:59.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:58 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:58.806+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:37:59.176 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:58 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:58.868+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:37:59.176 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:58 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:58.868+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:37:59.176 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:58 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:58.868+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:37:59.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:59 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:59.055+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:37:59.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:59 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:59.055+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:37:59.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:37:59 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:37:59.055+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:37:59.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:59 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:59.175+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:37:59.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:59 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:59.175+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:37:59.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:37:59 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:37:59.175+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:00.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:59 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:59.804+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:38:00.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:59 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:59.804+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:38:00.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:37:59 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:37:59.804+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:29.212344+0000 front 2023-12-14T18:37:29.212484+0000 (oldest deadline 2023-12-14T18:37:53.312304+0000) 2023-12-14T18:38:00.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:59 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:59.779+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:00.066 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:59 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:59.779+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:00.066 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:37:59 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:37:59.779+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:00.086 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:59 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:59.615+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:00.086 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:37:59 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:37:59.615+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:00.220 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:59 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:59.833+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:00.220 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:59 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:59.833+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:00.220 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:37:59 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:37:59.833+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:00.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:00 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:00.084+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:00.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:00 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:00.084+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:00.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:00 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:00.084+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:00.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:00 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:00.218+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:00.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:00 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:00.218+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:00.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:00 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:00.218+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:01.059 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:00 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:00.646+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:01.059 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:00 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:00.646+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:01.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:38:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:38:00.830+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:14.709655+0000 front 2023-12-14T18:37:14.709675+0000 (oldest deadline 2023-12-14T18:37:40.510519+0000) 2023-12-14T18:38:01.064 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:38:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:38:00.830+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.311667+0000 front 2023-12-14T18:37:23.311744+0000 (oldest deadline 2023-12-14T18:37:49.211784+0000) 2023-12-14T18:38:01.065 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:38:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6[65700]: 2023-12-14T18:38:00.830+0000 7fa7ce891700 -1 osd.6 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:29.212344+0000 front 2023-12-14T18:37:29.212484+0000 (oldest deadline 2023-12-14T18:37:53.312304+0000) 2023-12-14T18:38:01.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:00.744+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:01.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:00.744+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:01.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:00.744+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:01.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:00 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:00.744+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:01.192 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:00 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:00.787+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:01.192 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:00 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:00.787+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:01.193 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:00 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:00.787+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:01.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:01 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:01.057+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:01.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:01 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:01.057+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:01.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:01 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:01.057+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:01.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:01 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:01.057+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:01.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:01 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:01.192+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:01.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:01 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:01.192+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:01.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:01 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:01.192+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:01.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:01 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:01.738+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:01.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:01 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:01.738+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:01.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:01 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:01.738+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:02.013 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:01 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:01.628+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:02.014 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:01 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:01.628+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:02.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:01 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:01.698+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:02.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:01 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:01.698+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:02.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:01 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:01.698+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:02.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:01 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:01.698+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:02.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:02.012+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:02.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:02.012+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:02.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:02.012+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:02.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:02.012+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:02.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:02.232+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:02.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:02.232+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:02.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:02.232+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:02.565 INFO:journalctl@ceph.osd.6.smithi093.stdout:Dec 14 18:38:02 smithi093 bash[202309]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-6 2023-12-14T18:38:03.028 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:02.747+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:03.028 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:02.747+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:03.028 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:02.747+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:03.028 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:02 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:02.747+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:03.052 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:02.646+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:03.052 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:02.646+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:03.200 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:02.777+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:03.200 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:02.777+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:03.200 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:02 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:02.777+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:03.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:03.050+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:03.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:03.050+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:03.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:03.050+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:03.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:03.050+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:03.358 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.6.service' 2023-12-14T18:38:03.398 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:03.398 INFO:tasks.cephadm.osd.6:Stopped osd.6 2023-12-14T18:38:03.399 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2023-12-14T18:38:03.399 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.7 2023-12-14T18:38:03.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:03 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:03.199+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:03.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:03 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:03.199+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:03.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:03 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:03.199+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:03.843 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:03 smithi097 systemd[1]: Stopping Ceph osd.7 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:03.843 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:03.616+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:03.843 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:03.616+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:03.844 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:03.616+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.464639+0000 front 2023-12-14T18:37:38.464669+0000 (oldest deadline 2023-12-14T18:38:03.164646+0000) 2023-12-14T18:38:03.844 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:03.634+0000 7fb17d32b700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:03.844 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:03.634+0000 7fb17d32b700 -1 osd.7 101 *** Got signal Terminated *** 2023-12-14T18:38:03.844 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:03 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:03.634+0000 7fb17d32b700 -1 osd.7 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:38:04.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:03 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:03.757+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:04.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:03 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:03.757+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:04.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:03 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:03.757+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:04.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:03 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:03.757+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:04.184 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:03 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:03.767+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:04.184 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:03 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:03.767+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:04.184 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:03 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:03.767+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:04.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:04 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:04.051+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:04.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:04 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:04.051+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:04.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:04 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:04.051+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:04.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:04 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:04.051+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:04.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:04 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:04.181+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:04.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:04 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:04.181+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:04.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:04 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:04.181+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:04.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:04 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:04.733+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:04.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:04 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:04.733+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:04.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:04 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:04.733+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:05.045 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:04 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:04.645+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:05.045 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:04 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:04.645+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:05.045 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:04 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:04.645+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.464639+0000 front 2023-12-14T18:37:38.464669+0000 (oldest deadline 2023-12-14T18:38:03.164646+0000) 2023-12-14T18:38:05.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:04 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:04.737+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:05.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:04 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:04.737+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:05.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:04 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:04.737+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:05.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:04 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:04.737+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:05.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:05 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:05.043+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:05.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:05 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:05.043+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:05.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:05 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:05.043+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:05.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:05 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:05.043+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:05.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:05 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:05.231+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:05.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:05 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:05.231+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:05.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:05 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:05.231+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:05.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:05 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:05.231+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:05.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:05 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:05.702+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:05.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:05 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:05.702+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:05.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:05 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:05.702+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:06.016 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:05 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:05.687+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:06.016 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:05 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:05.687+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:06.016 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:05 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:05.687+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.464639+0000 front 2023-12-14T18:37:38.464669+0000 (oldest deadline 2023-12-14T18:38:03.164646+0000) 2023-12-14T18:38:06.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:05 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:05.738+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:06.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:05 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:05.738+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:06.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:05 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:05.738+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:06.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:05 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:05.738+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:06.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:06 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:06.015+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:06.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:06 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:06.015+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:06.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:06 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:06.015+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:06.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:06 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:06.015+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:06.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:06 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:06.199+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:06.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:06 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:06.199+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:06.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:06 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:06.199+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:06.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:06 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:06.199+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:06.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:06 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:06.662+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:06.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:06 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:06.662+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:06.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:06 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:06.662+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:07.011 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:06 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:06.642+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:07.011 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:06 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:06.642+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:07.011 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:06 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:06.642+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.464639+0000 front 2023-12-14T18:37:38.464669+0000 (oldest deadline 2023-12-14T18:38:03.164646+0000) 2023-12-14T18:38:07.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:06 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:06.733+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:07.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:06 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:06.733+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:07.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:06 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:06.733+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:07.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:06 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:06.733+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:07.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:07 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:07.010+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:07.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:07 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:07.010+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:07.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:07 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:07.010+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:07.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:07 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:07.010+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:07.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:07 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:07.235+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:07.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:07 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:07.235+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:07.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:07 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:07.235+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:07.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:07 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:07.235+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:07.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:07 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:07.676+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:07.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:07 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:07.676+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:07.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:07 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:07.676+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:08.026 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:07 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:07.633+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:08.026 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:07 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:07.633+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:08.026 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:07 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:07.633+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.464639+0000 front 2023-12-14T18:37:38.464669+0000 (oldest deadline 2023-12-14T18:38:03.164646+0000) 2023-12-14T18:38:08.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:07 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:07.755+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:08.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:07 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:07.755+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:08.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:07 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:07.755+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:08.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:07 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:07.755+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:08.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:08 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:08.024+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:08.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:08 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:08.024+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:08.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:08 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:08.024+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:08.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:08 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:08.024+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:08.721 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:08.269+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:08.721 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:08.269+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:08.721 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:08.269+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:08.722 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:08.269+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:08.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:08.719+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:08.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:08.719+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:08.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:08 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:08.720+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:09.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:08 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:08.795+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:09.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:08 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:08.795+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:09.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:08 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:08.795+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:09.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:08 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:08.795+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:09.066 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:08 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:08.626+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:23.460866+0000 front 2023-12-14T18:37:23.460888+0000 (oldest deadline 2023-12-14T18:37:48.761091+0000) 2023-12-14T18:38:09.066 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:08 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:08.626+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.562842+0000 front 2023-12-14T18:37:31.562730+0000 (oldest deadline 2023-12-14T18:37:56.262756+0000) 2023-12-14T18:38:09.066 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:08 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7[59365]: 2023-12-14T18:38:08.626+0000 7fb17791d700 -1 osd.7 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.464639+0000 front 2023-12-14T18:37:38.464669+0000 (oldest deadline 2023-12-14T18:38:03.164646+0000) 2023-12-14T18:38:09.322 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:09 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:09.065+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:09.322 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:09 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:09.065+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:09.322 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:09 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:09.065+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:09.322 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:09 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:09.065+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:09.323 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:09 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:09.065+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:09.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:09 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:09.220+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:09.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:09 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:09.220+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:09.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:09 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:09.220+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:09.496 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:09 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:09.220+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:09.592 INFO:journalctl@ceph.osd.7.smithi097.stdout:Dec 14 18:38:09 smithi097 bash[75219]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-7 2023-12-14T18:38:09.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:09 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:09.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:09.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:09 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:09.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:09.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:09 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:09.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:10.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:09.796+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:10.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:09.796+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:10.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:09.796+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:10.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:09.796+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:10.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:09 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:09.796+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:10.093 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:10 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:10.021+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:10.093 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:10 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:10.021+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:10.093 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:10 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:10.021+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:10.093 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:10 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:10.021+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:10.093 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:10 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:10.021+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:10.499 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.7.service' 2023-12-14T18:38:10.540 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:10.540 INFO:tasks.cephadm.osd.7:Stopped osd.7 2023-12-14T18:38:10.540 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2023-12-14T18:38:10.541 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.8 2023-12-14T18:38:10.572 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:10.246+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:10.572 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:10.246+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:10.573 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:10.246+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:10.573 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:10.246+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:10.996 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:10 smithi160 systemd[1]: Stopping Ceph osd.8 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:10.996 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:10.785+0000 7faec033d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:10.996 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:10.785+0000 7faec033d700 -1 osd.8 101 *** Got signal Terminated *** 2023-12-14T18:38:10.996 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:10.785+0000 7faec033d700 -1 osd.8 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:38:10.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:10.632+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:10.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:10.632+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:10.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:10.632+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:10.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:10 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:10.632+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:11.064 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:10.783+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:11.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:10.783+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:11.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:10.783+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:11.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:10.783+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:11.065 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:10 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:10.783+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:11.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:11 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:11.048+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:11.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:11 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:11.048+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:11.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:11 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:11.048+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:11.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:11 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:11.048+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:11.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:11 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:11.048+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:11.630 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:11.268+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:11.630 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:11.268+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:11.630 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:11.268+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:11.630 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:11.268+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:11.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:11.628+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:11.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:11.628+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:11.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:11.628+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:11.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:11.628+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:12.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:11 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:11.828+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:12.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:11 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:11.828+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:12.316 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:11 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:11.828+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:12.316 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:11 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:11.828+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:12.316 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:11 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:11.828+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:12.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:12 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:12.029+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:12.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:12 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:12.029+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:12.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:12 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:12.029+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:12.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:12 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:12.029+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:12.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:12 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:12.029+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:12.581 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:12 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:12.295+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:12.582 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:12 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:12.295+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:12.582 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:12 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:12.295+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:12.582 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:12 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:12.295+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:12.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:12 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:12.580+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:12.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:12 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:12.580+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:12.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:12 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:12.580+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:12.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:12 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:12.580+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:13.314 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:12.876+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:13.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:12.876+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:13.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:12.876+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:13.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:12.876+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:13.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:12 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:12.876+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:13.342 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:13 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:13.046+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:13.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:13 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:13.046+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:13.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:13 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:13.046+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:13.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:13 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:13.046+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:13.343 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:13 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:13.046+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:13.592 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:13 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:13.328+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:13.592 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:13 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:13.328+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:13.592 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:13 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:13.328+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:13.592 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:13 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:13.328+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:13.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:13 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:13.590+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:13.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:13 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:13.590+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:13.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:13 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:13.590+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:13.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:13 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:13.590+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:14.314 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:13 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:13.840+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:14.314 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:13 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:13.840+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:14.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:13 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:13.840+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:14.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:13 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:13.840+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:14.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:13 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:13.840+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:14.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:14 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:14.092+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:14.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:14 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:14.092+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:14.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:14 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:14.092+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:14.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:14 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:14.092+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:14.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:14 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:14.092+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:14.619 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:14 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:14.356+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:14.619 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:14 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:14.356+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:14.619 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:14 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:14.356+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:14.620 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:14 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:14.356+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:14.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:14 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:14.618+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:14.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:14 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:14.618+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:14.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:14 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:14.618+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:14.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:14 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:14.618+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:15.314 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:14 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:14.884+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:15.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:14 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:14.884+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:15.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:14 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:14.884+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:15.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:14 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:14.884+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:15.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:14 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:14.884+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:15.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:15 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:15.124+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:15.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:15 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:15.124+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:15.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:15 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:15.124+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:15.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:15 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:15.124+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:15.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:15 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:15.124+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:15.643 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:15.328+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:13.226256+0000 front 2023-12-14T18:37:13.226313+0000 (oldest deadline 2023-12-14T18:37:38.526460+0000) 2023-12-14T18:38:15.643 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:15.328+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:25.528189+0000 front 2023-12-14T18:37:25.528119+0000 (oldest deadline 2023-12-14T18:37:46.628214+0000) 2023-12-14T18:38:15.644 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:15.328+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.129229+0000 front 2023-12-14T18:37:30.129190+0000 (oldest deadline 2023-12-14T18:37:58.329983+0000) 2023-12-14T18:38:15.644 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:15.328+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:38.330823+0000 front 2023-12-14T18:37:38.330699+0000 (oldest deadline 2023-12-14T18:38:04.230459+0000) 2023-12-14T18:38:15.644 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8[60238]: 2023-12-14T18:38:15.328+0000 7faeb892b700 -1 osd.8 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:51.232059+0000 front 2023-12-14T18:37:51.232039+0000 (oldest deadline 2023-12-14T18:38:14.732125+0000) 2023-12-14T18:38:15.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:15.642+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:15.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:15.642+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:15.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:15.642+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:15.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:15.642+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:16.314 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:15 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:15.911+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:16.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:15 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:15.911+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:16.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:15 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:15.911+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:16.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:15 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:15.911+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:16.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:15 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:15.911+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:16.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:16 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:16.126+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:16.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:16 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:16.126+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:16.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:16 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:16.126+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:16.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:16 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:16.126+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:16.594 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:16 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:16.126+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:16.746 INFO:journalctl@ceph.osd.8.smithi160.stdout:Dec 14 18:38:16 smithi160 bash[76445]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-8 2023-12-14T18:38:16.746 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:16 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:16.609+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:16.747 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:16 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:16.609+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:16.747 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:16 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:16.609+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:16.747 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:16 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:16.609+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:17.314 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:16 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:16.922+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:17.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:16 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:16.922+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:17.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:16 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:16.922+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:17.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:16 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:16.923+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:17.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:16 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:16.923+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:17.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:17 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:17.169+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:17.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:17 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:17.169+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:17.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:17 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:17.169+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:17.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:17 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:17.169+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:17.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:17 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:17.169+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:17.725 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:17 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:17.644+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:17.725 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:17 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:17.644+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:17.725 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:17 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:17.644+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:17.725 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:17 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:17.644+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:17.732 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.8.service' 2023-12-14T18:38:17.811 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:17.812 INFO:tasks.cephadm.osd.8:Stopped osd.8 2023-12-14T18:38:17.812 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2023-12-14T18:38:17.812 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.9 2023-12-14T18:38:18.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:17 smithi093 systemd[1]: Stopping Ceph osd.9 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:18.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:17 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:17.968+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:18.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:17 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:17.968+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:18.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:17 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:17.968+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:18.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:17 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:17.968+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:18.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:17 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:17.968+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:18.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:17 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:17.968+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.622190+0000 front 2023-12-14T18:37:53.622179+0000 (oldest deadline 2023-12-14T18:38:17.122037+0000) 2023-12-14T18:38:18.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:18 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:18.066+0000 7fd7d8547700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:18.316 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:18 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:18.066+0000 7fd7d8547700 -1 osd.9 101 *** Got signal Terminated *** 2023-12-14T18:38:18.316 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:18 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:18.066+0000 7fd7d8547700 -1 osd.9 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:38:18.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:18 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:18.184+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:18.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:18 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:18.184+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:18.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:18 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:18.184+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:18.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:18 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:18.184+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:18.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:18 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:18.184+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:18.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:18 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:18.184+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:18.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:18 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:18.679+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:18.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:18 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:18.679+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:18.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:18 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:18.679+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:18.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:18 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:18.679+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:19.317 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:18 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:18.953+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:19.317 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:18 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:18.953+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:19.317 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:18 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:18.953+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:19.317 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:18 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:18.953+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:19.317 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:18 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:18.953+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:19.317 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:18 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:18.953+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.622190+0000 front 2023-12-14T18:37:53.622179+0000 (oldest deadline 2023-12-14T18:38:17.122037+0000) 2023-12-14T18:38:19.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:19 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:19.199+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:19.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:19 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:19.199+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:19.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:19 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:19.199+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:19.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:19 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:19.199+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:19.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:19 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:19.199+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:19.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:19 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:19.199+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:19.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:19 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:19.676+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:19.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:19 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:19.676+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:19.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:19 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:19.676+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:19.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:19 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:19.676+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:19.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:19 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:19.676+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:20.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:19.991+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:20.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:19.991+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:20.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:19.991+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:20.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:19.991+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:20.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:19.991+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:20.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:19 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:19.991+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.622190+0000 front 2023-12-14T18:37:53.622179+0000 (oldest deadline 2023-12-14T18:38:17.122037+0000) 2023-12-14T18:38:20.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:20 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:20.230+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:20.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:20 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:20.230+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:20.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:20 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:20.230+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:20.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:20 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:20.230+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:20.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:20 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:20.230+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:20.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:20 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:20.230+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:20.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:20 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:20.666+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:20.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:20 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:20.666+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:20.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:20 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:20.666+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:20.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:20 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:20.666+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:20.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:20 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:20.666+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:21.314 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:20 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:20.941+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:21.314 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:20 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:20.941+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:21.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:20 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:20.941+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:21.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:20 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:20.941+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:21.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:20 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:20.941+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:21.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:20 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:20.941+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.622190+0000 front 2023-12-14T18:37:53.622179+0000 (oldest deadline 2023-12-14T18:38:17.122037+0000) 2023-12-14T18:38:21.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:21.246+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:21.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:21.246+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:21.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:21.246+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:21.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:21.246+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:21.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:21.246+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:21.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:21 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:21.246+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:21.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:21.639+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:21.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:21.639+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:21.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:21.639+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:21.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:21.639+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:21.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:21 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:21.639+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:22.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:21.981+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:22.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:21.981+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:22.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:21.981+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:22.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:21.981+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:22.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:21.981+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:22.316 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:21 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:21.981+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.622190+0000 front 2023-12-14T18:37:53.622179+0000 (oldest deadline 2023-12-14T18:38:17.122037+0000) 2023-12-14T18:38:22.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:22.204+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:22.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:22.204+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:22.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:22.204+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:22.594 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:22.204+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:22.594 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:22.204+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:22.594 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:22 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:22.204+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:22.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:22.627+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:22.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:22.627+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:22.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:22.627+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:22.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:22.627+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:22.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:22.627+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:22.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:22 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:22.627+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:23.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:22.962+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.016550+0000 front 2023-12-14T18:37:15.016638+0000 (oldest deadline 2023-12-14T18:37:39.716695+0000) 2023-12-14T18:38:23.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:22.962+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:20.817559+0000 front 2023-12-14T18:37:20.817661+0000 (oldest deadline 2023-12-14T18:37:46.117569+0000) 2023-12-14T18:38:23.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:22.962+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:30.218704+0000 front 2023-12-14T18:37:30.218815+0000 (oldest deadline 2023-12-14T18:37:53.718769+0000) 2023-12-14T18:38:23.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:22.962+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:34.819681+0000 front 2023-12-14T18:37:34.819693+0000 (oldest deadline 2023-12-14T18:38:00.719843+0000) 2023-12-14T18:38:23.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:22.962+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:43.621028+0000 front 2023-12-14T18:37:43.621083+0000 (oldest deadline 2023-12-14T18:38:09.520989+0000) 2023-12-14T18:38:23.315 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:22 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9[71090]: 2023-12-14T18:38:22.962+0000 7fd7d333a700 -1 osd.9 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.622190+0000 front 2023-12-14T18:37:53.622179+0000 (oldest deadline 2023-12-14T18:38:17.122037+0000) 2023-12-14T18:38:23.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:23.202+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:23.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:23.202+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:23.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:23.202+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:23.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:23.202+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:23.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:23.202+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:23.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:23 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:23.202+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:23.815 INFO:journalctl@ceph.osd.9.smithi093.stdout:Dec 14 18:38:23 smithi093 bash[202563]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-9 2023-12-14T18:38:23.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:23.668+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:23.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:23.668+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:23.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:23.668+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:23.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:23.668+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:23.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:23.668+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:23.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:23 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:23.668+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:24.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:24.226+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:24.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:24.226+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:24.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:24.226+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:24.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:24.226+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:24.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:24.226+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:24.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:24.226+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:24.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:24 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:24.226+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:59.170402+0000 front 2023-12-14T18:37:59.170310+0000 (oldest deadline 2023-12-14T18:38:23.270247+0000) 2023-12-14T18:38:24.763 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.9.service' 2023-12-14T18:38:24.803 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:24.804 INFO:tasks.cephadm.osd.9:Stopped osd.9 2023-12-14T18:38:24.804 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2023-12-14T18:38:24.804 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.10 2023-12-14T18:38:24.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:24 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:24.656+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:24.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:24 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:24.656+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:24.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:24 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:24.656+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:24.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:24 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:24.656+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:24.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:24 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:24.656+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:24.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:24 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:24.656+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:25.092 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:24 smithi097 systemd[1]: Stopping Ceph osd.10 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:25.092 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.049+0000 7fee83bc2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:25.092 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.049+0000 7fee83bc2700 -1 osd.10 101 *** Got signal Terminated *** 2023-12-14T18:38:25.092 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.049+0000 7fee83bc2700 -1 osd.10 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:38:25.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.212+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:25.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.212+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:25.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.212+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:25.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.212+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:25.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.212+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:25.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.212+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:25.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:25 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:25.212+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:59.170402+0000 front 2023-12-14T18:37:59.170310+0000 (oldest deadline 2023-12-14T18:38:23.270247+0000) 2023-12-14T18:38:25.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:25.655+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:25.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:25.655+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:25.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:25.655+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:25.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:25.655+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:25.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:25.655+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:25.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:25 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:25.655+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:26.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:26 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:26.178+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:26.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:26 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:26.178+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:26.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:26 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:26.178+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:26.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:26 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:26.178+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:26.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:26 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:26.178+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:26.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:26 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:26.178+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:26.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:26 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:26.178+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:59.170402+0000 front 2023-12-14T18:37:59.170310+0000 (oldest deadline 2023-12-14T18:38:23.270247+0000) 2023-12-14T18:38:26.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:26.664+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:26.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:26.664+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:26.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:26.664+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:26.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:26.664+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:26.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:26.664+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:26.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:26 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:26.664+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:27.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:27.171+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:27.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:27.171+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:27.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:27.171+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:27.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:27.171+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:27.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:27.171+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:27.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:27.171+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:27.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:27 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:27.171+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:59.170402+0000 front 2023-12-14T18:37:59.170310+0000 (oldest deadline 2023-12-14T18:38:23.270247+0000) 2023-12-14T18:38:27.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:27.701+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:27.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:27.701+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:27.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:27.701+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:27.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:27.701+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:27.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:27.701+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:27.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:27 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:27.701+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:28.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:28.182+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:28.592 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:28.182+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:28.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:28.182+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:28.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:28.182+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:28.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:28.182+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:28.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:28.182+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:28.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:28 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:28.182+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:59.170402+0000 front 2023-12-14T18:37:59.170310+0000 (oldest deadline 2023-12-14T18:38:23.270247+0000) 2023-12-14T18:38:28.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:28.731+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:28.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:28.731+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:28.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:28.731+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:28.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:28.731+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:28.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:28.731+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:28.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:28 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:28.731+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:29.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:29.139+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.562854+0000 front 2023-12-14T18:37:15.562840+0000 (oldest deadline 2023-12-14T18:37:39.062946+0000) 2023-12-14T18:38:29.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:29.139+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:22.464771+0000 front 2023-12-14T18:37:22.464802+0000 (oldest deadline 2023-12-14T18:37:47.764936+0000) 2023-12-14T18:38:29.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:29.139+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:31.265715+0000 front 2023-12-14T18:37:31.265797+0000 (oldest deadline 2023-12-14T18:37:54.165853+0000) 2023-12-14T18:38:29.593 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:29.139+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6814 osd.3 since back 2023-12-14T18:37:35.866998+0000 front 2023-12-14T18:37:35.867009+0000 (oldest deadline 2023-12-14T18:38:00.566849+0000) 2023-12-14T18:38:29.594 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:29.139+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:45.867776+0000 front 2023-12-14T18:37:45.867745+0000 (oldest deadline 2023-12-14T18:38:08.167865+0000) 2023-12-14T18:38:29.594 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:29.139+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:53.369609+0000 front 2023-12-14T18:37:53.369448+0000 (oldest deadline 2023-12-14T18:38:18.069441+0000) 2023-12-14T18:38:29.594 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:29 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10[64789]: 2023-12-14T18:38:29.139+0000 7fee7d9b3700 -1 osd.10 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:59.170402+0000 front 2023-12-14T18:37:59.170310+0000 (oldest deadline 2023-12-14T18:38:23.270247+0000) 2023-12-14T18:38:30.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:29 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:29.764+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:30.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:29 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:29.764+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:30.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:29 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:29.764+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:30.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:29 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:29.764+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:30.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:29 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:29.764+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:30.247 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:29 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:29.764+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:31.094 INFO:journalctl@ceph.osd.10.smithi097.stdout:Dec 14 18:38:30 smithi097 bash[75484]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-10 2023-12-14T18:38:31.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:30 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:30.794+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:31.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:30 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:30.794+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:31.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:30 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:30.794+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:31.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:30 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:30.794+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:31.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:30 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:30.794+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:31.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:30 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:30.794+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:31.837 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.10.service' 2023-12-14T18:38:31.876 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:31.877 INFO:tasks.cephadm.osd.10:Stopped osd.10 2023-12-14T18:38:31.877 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2023-12-14T18:38:31.877 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.11 2023-12-14T18:38:32.150 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:31.752+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:32.150 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:31.752+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:32.150 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:31.752+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:32.151 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:31.752+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:32.151 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:31.752+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:32.151 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:31.752+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:32.151 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:31 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:31.752+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2023-12-14T18:38:05.764947+0000 front 2023-12-14T18:38:05.764975+0000 (oldest deadline 2023-12-14T18:38:31.064894+0000) 2023-12-14T18:38:32.151 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:31 smithi160 systemd[1]: Stopping Ceph osd.11 for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:32.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.149+0000 7fd6b5f61700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:32.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.149+0000 7fd6b5f61700 -1 osd.11 101 *** Got signal Terminated *** 2023-12-14T18:38:32.496 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.149+0000 7fd6b5f61700 -1 osd.11 101 *** Immediate shutdown (osd_fast_shutdown=true) *** 2023-12-14T18:38:33.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.755+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:33.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.755+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:33.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.755+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:33.246 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.755+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:33.247 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.755+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:33.247 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.755+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:33.247 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:32 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:32.755+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2023-12-14T18:38:05.764947+0000 front 2023-12-14T18:38:05.764975+0000 (oldest deadline 2023-12-14T18:38:31.064894+0000) 2023-12-14T18:38:33.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:33 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:33.730+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:33.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:33 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:33.730+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:33.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:33 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:33.730+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:33.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:33 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:33.730+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:33.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:33 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:33.730+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:33.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:33 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:33.730+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:33.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:33 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:33.730+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2023-12-14T18:38:05.764947+0000 front 2023-12-14T18:38:05.764975+0000 (oldest deadline 2023-12-14T18:38:31.064894+0000) 2023-12-14T18:38:34.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:34 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:34.707+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:34.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:34 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:34.707+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:34.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:34 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:34.707+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:34.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:34 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:34.707+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:34.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:34 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:34.707+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:34.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:34 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:34.707+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:34.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:34 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:34.708+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2023-12-14T18:38:05.764947+0000 front 2023-12-14T18:38:05.764975+0000 (oldest deadline 2023-12-14T18:38:31.064894+0000) 2023-12-14T18:38:35.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:35 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:35.661+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:35.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:35 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:35.661+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:35.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:35 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:35.661+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:35.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:35 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:35.661+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:35.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:35 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:35.661+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:35.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:35 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:35.661+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:35.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:35 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:35.661+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2023-12-14T18:38:05.764947+0000 front 2023-12-14T18:38:05.764975+0000 (oldest deadline 2023-12-14T18:38:31.064894+0000) 2023-12-14T18:38:36.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:36 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:36.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6806 osd.0 since back 2023-12-14T18:37:15.557009+0000 front 2023-12-14T18:37:15.557029+0000 (oldest deadline 2023-12-14T18:37:39.057030+0000) 2023-12-14T18:38:36.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:36 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:36.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6804 osd.1 since back 2023-12-14T18:37:24.958367+0000 front 2023-12-14T18:37:24.958431+0000 (oldest deadline 2023-12-14T18:37:46.658341+0000) 2023-12-14T18:38:36.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:36 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:36.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6804 osd.2 since back 2023-12-14T18:37:32.360447+0000 front 2023-12-14T18:37:32.360427+0000 (oldest deadline 2023-12-14T18:37:57.060611+0000) 2023-12-14T18:38:36.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:36 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:36.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6812 osd.4 since back 2023-12-14T18:37:46.461914+0000 front 2023-12-14T18:37:46.461879+0000 (oldest deadline 2023-12-14T18:38:10.562052+0000) 2023-12-14T18:38:36.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:36 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:36.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.160:6812 osd.5 since back 2023-12-14T18:37:50.562575+0000 front 2023-12-14T18:37:50.562598+0000 (oldest deadline 2023-12-14T18:38:18.763216+0000) 2023-12-14T18:38:36.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:36 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:36.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.93:6822 osd.6 since back 2023-12-14T18:37:58.764034+0000 front 2023-12-14T18:37:58.764049+0000 (oldest deadline 2023-12-14T18:38:22.263991+0000) 2023-12-14T18:38:36.997 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:36 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11[65736]: 2023-12-14T18:38:36.675+0000 7fd6b1555700 -1 osd.11 101 heartbeat_check: no reply from 172.21.15.97:6820 osd.7 since back 2023-12-14T18:38:05.764947+0000 front 2023-12-14T18:38:05.764975+0000 (oldest deadline 2023-12-14T18:38:31.064894+0000) 2023-12-14T18:38:37.996 INFO:journalctl@ceph.osd.11.smithi160.stdout:Dec 14 18:38:37 smithi160 bash[76700]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-osd-11 2023-12-14T18:38:38.931 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@osd.11.service' 2023-12-14T18:38:38.971 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:38.972 INFO:tasks.cephadm.osd.11:Stopped osd.11 2023-12-14T18:38:38.972 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2023-12-14T18:38:38.972 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.a 2023-12-14T18:38:39.290 INFO:journalctl@ceph.mds.a.smithi093.stdout:Dec 14 18:38:39 smithi093 systemd[1]: Stopping Ceph mds.a for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:39.564 INFO:journalctl@ceph.mds.a.smithi093.stdout:Dec 14 18:38:39 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-a[76307]: 2023-12-14T18:38:39.289+0000 7f1a57664700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:39.564 INFO:journalctl@ceph.mds.a.smithi093.stdout:Dec 14 18:38:39 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-a[76307]: 2023-12-14T18:38:39.289+0000 7f1a57664700 -1 mds.a *** got signal Terminated *** 2023-12-14T18:38:41.974 INFO:journalctl@ceph.mds.a.smithi093.stdout:Dec 14 18:38:41 smithi093 bash[202820]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-a 2023-12-14T18:38:41.980 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.a.service' 2023-12-14T18:38:42.016 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:42.016 INFO:tasks.cephadm.mds.a:Stopped mds.a 2023-12-14T18:38:42.016 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2023-12-14T18:38:42.016 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.d 2023-12-14T18:38:42.272 INFO:journalctl@ceph.mds.d.smithi093.stdout:Dec 14 18:38:42 smithi093 systemd[1]: Stopping Ceph mds.d for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:42.564 INFO:journalctl@ceph.mds.d.smithi093.stdout:Dec 14 18:38:42 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-d[75399]: 2023-12-14T18:38:42.271+0000 7f303a45e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:42.564 INFO:journalctl@ceph.mds.d.smithi093.stdout:Dec 14 18:38:42 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-d[75399]: 2023-12-14T18:38:42.271+0000 7f303a45e700 -1 mds.d *** got signal Terminated *** 2023-12-14T18:38:45.189 INFO:journalctl@ceph.mds.d.smithi093.stdout:Dec 14 18:38:44 smithi093 bash[202958]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-d 2023-12-14T18:38:45.194 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.d.service' 2023-12-14T18:38:45.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:45.231 INFO:tasks.cephadm.mds.d:Stopped mds.d 2023-12-14T18:38:45.231 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2023-12-14T18:38:45.231 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.g 2023-12-14T18:38:45.489 INFO:journalctl@ceph.mds.g.smithi093.stdout:Dec 14 18:38:45 smithi093 systemd[1]: Stopping Ceph mds.g for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:45.814 INFO:journalctl@ceph.mds.g.smithi093.stdout:Dec 14 18:38:45 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-g[75921]: 2023-12-14T18:38:45.488+0000 7f6156085700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:45.814 INFO:journalctl@ceph.mds.g.smithi093.stdout:Dec 14 18:38:45 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-g[75921]: 2023-12-14T18:38:45.488+0000 7f6156085700 -1 mds.g *** got signal Terminated *** 2023-12-14T18:38:48.166 INFO:journalctl@ceph.mds.g.smithi093.stdout:Dec 14 18:38:47 smithi093 bash[203105]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-g 2023-12-14T18:38:48.173 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.g.service' 2023-12-14T18:38:48.207 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:48.207 INFO:tasks.cephadm.mds.g:Stopped mds.g 2023-12-14T18:38:48.207 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2023-12-14T18:38:48.208 DEBUG:teuthology.orchestra.run.smithi093:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.j 2023-12-14T18:38:48.464 INFO:journalctl@ceph.mds.j.smithi093.stdout:Dec 14 18:38:48 smithi093 systemd[1]: Stopping Ceph mds.j for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:48.814 INFO:journalctl@ceph.mds.j.smithi093.stdout:Dec 14 18:38:48 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-j[75012]: 2023-12-14T18:38:48.463+0000 7f4eb2334700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:48.814 INFO:journalctl@ceph.mds.j.smithi093.stdout:Dec 14 18:38:48 smithi093 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-j[75012]: 2023-12-14T18:38:48.463+0000 7f4eb2334700 -1 mds.j *** got signal Terminated *** 2023-12-14T18:38:51.530 INFO:journalctl@ceph.mds.j.smithi093.stdout:Dec 14 18:38:51 smithi093 bash[203243]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-j 2023-12-14T18:38:51.536 DEBUG:teuthology.orchestra.run.smithi093:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.j.service' 2023-12-14T18:38:51.569 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:51.569 INFO:tasks.cephadm.mds.j:Stopped mds.j 2023-12-14T18:38:51.569 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2023-12-14T18:38:51.570 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.b 2023-12-14T18:38:52.092 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:38:51 smithi097 systemd[1]: Stopping Ceph mds.b for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:52.092 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:38:51 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-b[68560]: 2023-12-14T18:38:51.811+0000 7f80079db700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:52.092 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:38:51 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-b[68560]: 2023-12-14T18:38:51.811+0000 7f80079db700 -1 mds.b *** got signal Terminated *** 2023-12-14T18:38:54.592 INFO:journalctl@ceph.mds.b.smithi097.stdout:Dec 14 18:38:54 smithi097 bash[75738]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-b 2023-12-14T18:38:54.649 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.b.service' 2023-12-14T18:38:54.684 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:54.685 INFO:tasks.cephadm.mds.b:Stopped mds.b 2023-12-14T18:38:54.685 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2023-12-14T18:38:54.685 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.e 2023-12-14T18:38:54.946 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:38:54 smithi097 systemd[1]: Stopping Ceph mds.e for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:55.342 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:38:54 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-e[67879]: 2023-12-14T18:38:54.943+0000 7f8466d09700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:55.343 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:38:54 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-e[67879]: 2023-12-14T18:38:54.943+0000 7f8466d09700 -1 mds.e *** got signal Terminated *** 2023-12-14T18:38:58.593 INFO:journalctl@ceph.mds.e.smithi097.stdout:Dec 14 18:38:58 smithi097 bash[75876]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-e 2023-12-14T18:38:58.641 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.e.service' 2023-12-14T18:38:58.676 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:38:58.676 INFO:tasks.cephadm.mds.e:Stopped mds.e 2023-12-14T18:38:58.676 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2023-12-14T18:38:58.676 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.h 2023-12-14T18:38:58.936 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:38:58 smithi097 systemd[1]: Stopping Ceph mds.h for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:38:59.342 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:38:58 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-h[68220]: 2023-12-14T18:38:58.934+0000 7f6a5d8c3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:38:59.343 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:38:58 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-h[68220]: 2023-12-14T18:38:58.934+0000 7f6a5d8c3700 -1 mds.h *** got signal Terminated *** 2023-12-14T18:39:02.591 INFO:journalctl@ceph.mds.h.smithi097.stdout:Dec 14 18:39:02 smithi097 bash[76014]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-h 2023-12-14T18:39:02.596 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.h.service' 2023-12-14T18:39:02.629 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:02.630 INFO:tasks.cephadm.mds.h:Stopped mds.h 2023-12-14T18:39:02.630 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2023-12-14T18:39:02.630 DEBUG:teuthology.orchestra.run.smithi097:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.k 2023-12-14T18:39:02.895 INFO:journalctl@ceph.mds.k.smithi097.stdout:Dec 14 18:39:02 smithi097 systemd[1]: Stopping Ceph mds.k for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:03.342 INFO:journalctl@ceph.mds.k.smithi097.stdout:Dec 14 18:39:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-k[67537]: 2023-12-14T18:39:02.893+0000 7fe69aac2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:03.343 INFO:journalctl@ceph.mds.k.smithi097.stdout:Dec 14 18:39:02 smithi097 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-k[67537]: 2023-12-14T18:39:02.893+0000 7fe69aac2700 -1 mds.k *** got signal Terminated *** 2023-12-14T18:39:05.842 INFO:journalctl@ceph.mds.k.smithi097.stdout:Dec 14 18:39:05 smithi097 bash[76153]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-k 2023-12-14T18:39:05.864 DEBUG:teuthology.orchestra.run.smithi097:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.k.service' 2023-12-14T18:39:05.897 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:05.897 INFO:tasks.cephadm.mds.k:Stopped mds.k 2023-12-14T18:39:05.898 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2023-12-14T18:39:05.898 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.c 2023-12-14T18:39:06.190 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:39:05 smithi160 systemd[1]: Stopping Ceph mds.c for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:06.496 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:39:06 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-c[69302]: 2023-12-14T18:39:06.188+0000 7f2e9b72d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:06.496 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:39:06 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-c[69302]: 2023-12-14T18:39:06.188+0000 7f2e9b72d700 -1 mds.c *** got signal Terminated *** 2023-12-14T18:39:10.681 INFO:journalctl@ceph.mds.c.smithi160.stdout:Dec 14 18:39:10 smithi160 bash[76963]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-c 2023-12-14T18:39:10.687 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.c.service' 2023-12-14T18:39:10.723 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:10.724 INFO:tasks.cephadm.mds.c:Stopped mds.c 2023-12-14T18:39:10.724 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2023-12-14T18:39:10.724 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.f 2023-12-14T18:39:10.996 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:39:10 smithi160 systemd[1]: Stopping Ceph mds.f for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:11.496 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:39:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-f[68962]: 2023-12-14T18:39:11.009+0000 7fda6d30a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:11.496 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:39:11 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-f[68962]: 2023-12-14T18:39:11.009+0000 7fda6d30a700 -1 mds.f *** got signal Terminated *** 2023-12-14T18:39:14.697 INFO:journalctl@ceph.mds.f.smithi160.stdout:Dec 14 18:39:14 smithi160 bash[77105]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-f 2023-12-14T18:39:14.702 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.f.service' 2023-12-14T18:39:14.737 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:14.737 INFO:tasks.cephadm.mds.f:Stopped mds.f 2023-12-14T18:39:14.737 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2023-12-14T18:39:14.737 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.i 2023-12-14T18:39:14.996 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:39:14 smithi160 systemd[1]: Stopping Ceph mds.i for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:15.495 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:39:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-i[69643]: 2023-12-14T18:39:15.007+0000 7f4d09fa3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:15.496 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:39:15 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-i[69643]: 2023-12-14T18:39:15.007+0000 7f4d09fa3700 -1 mds.i *** got signal Terminated *** 2023-12-14T18:39:18.639 INFO:journalctl@ceph.mds.i.smithi160.stdout:Dec 14 18:39:18 smithi160 bash[77243]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-i 2023-12-14T18:39:18.643 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.i.service' 2023-12-14T18:39:18.689 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:18.689 INFO:tasks.cephadm.mds.i:Stopped mds.i 2023-12-14T18:39:18.689 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2023-12-14T18:39:18.690 DEBUG:teuthology.orchestra.run.smithi160:> sudo systemctl stop ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.l 2023-12-14T18:39:18.967 INFO:journalctl@ceph.mds.l.smithi160.stdout:Dec 14 18:39:18 smithi160 systemd[1]: Stopping Ceph mds.l for 1c5ac00e-9aad-11ee-95a5-87774f69a715... 2023-12-14T18:39:19.245 INFO:journalctl@ceph.mds.l.smithi160.stdout:Dec 14 18:39:18 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-l[68618]: 2023-12-14T18:39:18.965+0000 7f6a95a16700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2023-12-14T18:39:19.246 INFO:journalctl@ceph.mds.l.smithi160.stdout:Dec 14 18:39:18 smithi160 ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-l[68618]: 2023-12-14T18:39:18.965+0000 7f6a95a16700 -1 mds.l *** got signal Terminated *** 2023-12-14T18:39:23.492 INFO:journalctl@ceph.mds.l.smithi160.stdout:Dec 14 18:39:23 smithi160 bash[77382]: ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715-mds-l 2023-12-14T18:39:23.498 DEBUG:teuthology.orchestra.run.smithi160:> sudo pkill -f 'journalctl -f -n 0 -u ceph-1c5ac00e-9aad-11ee-95a5-87774f69a715@mds.l.service' 2023-12-14T18:39:23.531 DEBUG:teuthology.orchestra.run:got remote process result: None 2023-12-14T18:39:23.531 INFO:tasks.cephadm.mds.l:Stopped mds.l 2023-12-14T18:39:23.531 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 --force --keep-logs 2023-12-14T18:39:25.818 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 --force --keep-logs 2023-12-14T18:39:27.582 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 --force --keep-logs 2023-12-14T18:39:29.253 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:39:29.291 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:39:29.328 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2023-12-14T18:39:29.358 INFO:tasks.cephadm:Archiving crash dumps... 2023-12-14T18:39:29.359 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/remote/smithi093/crash 2023-12-14T18:39:29.360 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/crash -- . 2023-12-14T18:39:29.405 INFO:teuthology.orchestra.run.smithi093.stderr:tar: /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/crash: Cannot open: No such file or directory 2023-12-14T18:39:29.405 INFO:teuthology.orchestra.run.smithi093.stderr:tar: Error is not recoverable: exiting now 2023-12-14T18:39:29.407 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/remote/smithi097/crash 2023-12-14T18:39:29.408 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/crash -- . 2023-12-14T18:39:29.468 INFO:teuthology.orchestra.run.smithi097.stderr:tar: /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/crash: Cannot open: No such file or directory 2023-12-14T18:39:29.469 INFO:teuthology.orchestra.run.smithi097.stderr:tar: Error is not recoverable: exiting now 2023-12-14T18:39:29.470 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/crash to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/remote/smithi160/crash 2023-12-14T18:39:29.471 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/crash -- . 2023-12-14T18:39:29.535 INFO:teuthology.orchestra.run.smithi160.stderr:tar: /var/lib/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/crash: Cannot open: No such file or directory 2023-12-14T18:39:29.535 INFO:teuthology.orchestra.run.smithi160.stderr:tar: Error is not recoverable: exiting now 2023-12-14T18:39:29.536 INFO:tasks.cephadm:Checking cluster log for badness... 2023-12-14T18:39:29.536 DEBUG:teuthology.orchestra.run.smithi093:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-14T18:39:29.573 INFO:teuthology.orchestra.run.smithi093.stderr:grep: /var/log/ceph/1c5ac00e-9aad-11ee-95a5-87774f69a715/ceph.log: No such file or directory 2023-12-14T18:39:29.576 INFO:tasks.cephadm:Compressing logs... 2023-12-14T18:39:29.576 DEBUG:teuthology.orchestra.run.smithi093:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:39:29.619 DEBUG:teuthology.orchestra.run.smithi097:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:39:29.621 DEBUG:teuthology.orchestra.run.smithi160:> sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:39:29.647 INFO:teuthology.orchestra.run.smithi160.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-14T18:39:29.657 INFO:teuthology.orchestra.run.smithi093.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-14T18:39:29.659 INFO:teuthology.orchestra.run.smithi097.stderr:find: ‘/var/log/rbd-target-api’: No such file or directory 2023-12-14T18:48:18.821 INFO:tasks.cephadm:Archiving logs... 2023-12-14T18:48:18.822 DEBUG:teuthology.misc:Transferring archived files from smithi093:/var/log/ceph to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/remote/smithi093/log 2023-12-14T18:48:18.823 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-14T18:48:30.258 DEBUG:teuthology.misc:Transferring archived files from smithi097:/var/log/ceph to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/remote/smithi097/log 2023-12-14T18:48:30.259 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-14T18:49:10.956 DEBUG:teuthology.misc:Transferring archived files from smithi160:/var/log/ceph to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/remote/smithi160/log 2023-12-14T18:49:10.957 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-14T18:49:39.788 INFO:tasks.cephadm:Removing cluster... 2023-12-14T18:49:39.790 DEBUG:teuthology.orchestra.run.smithi093:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 --force 2023-12-14T18:49:40.484 DEBUG:teuthology.orchestra.run.smithi097:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 --force 2023-12-14T18:49:41.344 DEBUG:teuthology.orchestra.run.smithi160:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 1c5ac00e-9aad-11ee-95a5-87774f69a715 --force 2023-12-14T18:49:42.128 INFO:tasks.cephadm:Removing cephadm ... 2023-12-14T18:49:42.129 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-14T18:49:42.151 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-14T18:49:42.184 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf /home/ubuntu/cephtest/cephadm 2023-12-14T18:49:42.202 INFO:tasks.cephadm:Teardown complete 2023-12-14T18:49:42.203 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-14T18:49:42.216 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-14T18:49:42.216 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-14T18:49:42.219 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-14T18:49:42.228 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-14T18:49:42.344 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-14T18:49:42.355 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-14T18:49:42.355 DEBUG:teuthology.orchestra.run.smithi093:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:49:42.358 DEBUG:teuthology.orchestra.run.smithi097:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:49:42.361 DEBUG:teuthology.orchestra.run.smithi160:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-14T18:49:42.375 INFO:teuthology.orchestra.run.smithi093.stderr:bash: ntpq: command not found 2023-12-14T18:49:42.376 INFO:teuthology.orchestra.run.smithi097.stderr:bash: ntpq: command not found 2023-12-14T18:49:42.377 INFO:teuthology.orchestra.run.smithi160.stderr:bash: ntpq: command not found 2023-12-14T18:49:42.382 INFO:teuthology.orchestra.run.smithi160.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:49:42.382 INFO:teuthology.orchestra.run.smithi160.stdout:=============================================================================== 2023-12-14T18:49:42.382 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 61 +499us[ +499us] +/- 82ms 2023-12-14T18:49:42.382 INFO:teuthology.orchestra.run.smithi160.stdout:^* hv02.front.sepia.ceph.com 3 6 377 57 -681us[ -681us] +/- 52ms 2023-12-14T18:49:42.382 INFO:teuthology.orchestra.run.smithi160.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 62 +534us[ +535us] +/- 75ms 2023-12-14T18:49:42.383 INFO:teuthology.orchestra.run.smithi160.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:49:42.399 INFO:teuthology.orchestra.run.smithi097.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:49:42.400 INFO:teuthology.orchestra.run.smithi097.stdout:=============================================================================== 2023-12-14T18:49:42.400 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 57 +457us[ +457us] +/- 82ms 2023-12-14T18:49:42.400 INFO:teuthology.orchestra.run.smithi097.stdout:^* hv02.front.sepia.ceph.com 3 6 377 62 -729us[ -741us] +/- 52ms 2023-12-14T18:49:42.400 INFO:teuthology.orchestra.run.smithi097.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 61 +479us[ +479us] +/- 75ms 2023-12-14T18:49:42.400 INFO:teuthology.orchestra.run.smithi097.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:49:42.662 INFO:teuthology.orchestra.run.smithi093.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-12-14T18:49:42.662 INFO:teuthology.orchestra.run.smithi093.stdout:=============================================================================== 2023-12-14T18:49:42.662 INFO:teuthology.orchestra.run.smithi093.stdout:^+ hv01.front.sepia.ceph.com 2 7 377 65 +504us[ +502us] +/- 82ms 2023-12-14T18:49:42.662 INFO:teuthology.orchestra.run.smithi093.stdout:^* hv02.front.sepia.ceph.com 3 6 377 61 -678us[ -680us] +/- 52ms 2023-12-14T18:49:42.662 INFO:teuthology.orchestra.run.smithi093.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 60 +522us[ +522us] +/- 75ms 2023-12-14T18:49:42.663 INFO:teuthology.orchestra.run.smithi093.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2023-12-14T18:49:42.663 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-14T18:49:42.674 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-14T18:49:42.675 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-14T18:49:42.712 DEBUG:teuthology.orchestra.run.smithi093:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-14T18:49:42.878 DEBUG:teuthology.orchestra.run.smithi097:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-14T18:49:42.972 DEBUG:teuthology.orchestra.run.smithi160:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-12-14T18:49:43.035 DEBUG:teuthology.orchestra.run.smithi093:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:49:43.071 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:49:43.071 DEBUG:teuthology.orchestra.run.smithi097:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:49:43.104 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:49:43.105 DEBUG:teuthology.orchestra.run.smithi160:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-12-14T18:49:43.134 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:49:43.135 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-14T18:49:43.147 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-14T18:49:43.192 INFO:teuthology.task.internal:Duration was 2570.675473 seconds 2023-12-14T18:49:43.192 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-14T18:49:43.207 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-14T18:49:43.208 DEBUG:teuthology.orchestra.run.smithi093:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-14T18:49:43.211 DEBUG:teuthology.orchestra.run.smithi097:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-14T18:49:43.213 DEBUG:teuthology.orchestra.run.smithi160:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-14T18:49:43.259 INFO:teuthology.orchestra.run.smithi160.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:49:43.313 INFO:teuthology.orchestra.run.smithi093.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:49:43.336 INFO:teuthology.orchestra.run.smithi097.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-12-14T18:49:43.616 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-14T18:49:43.617 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi093.front.sepia.ceph.com 2023-12-14T18:49:43.617 DEBUG:teuthology.orchestra.run.smithi093:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-14T18:49:43.654 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi097.front.sepia.ceph.com 2023-12-14T18:49:43.654 DEBUG:teuthology.orchestra.run.smithi097:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-14T18:49:43.686 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi160.front.sepia.ceph.com 2023-12-14T18:49:43.686 DEBUG:teuthology.orchestra.run.smithi160:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-14T18:49:43.709 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-14T18:49:43.709 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:49:43.712 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:49:43.729 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-14T18:49:43.791 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-14T18:49:43.791 DEBUG:teuthology.orchestra.run.smithi093:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-14T18:49:43.794 DEBUG:teuthology.orchestra.run.smithi097:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-14T18:49:43.814 DEBUG:teuthology.orchestra.run.smithi160:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-14T18:49:43.906 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-14T18:49:43.918 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-14T18:49:43.918 DEBUG:teuthology.orchestra.run.smithi093:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-14T18:49:43.946 DEBUG:teuthology.orchestra.run.smithi097:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-14T18:49:43.987 DEBUG:teuthology.orchestra.run.smithi160:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-14T18:49:44.015 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-14T18:49:44.027 DEBUG:teuthology.orchestra.run.smithi093:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:49:44.030 DEBUG:teuthology.orchestra.run.smithi097:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:49:44.033 DEBUG:teuthology.orchestra.run.smithi160:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:49:44.060 INFO:teuthology.orchestra.run.smithi097.stdout:kernel.core_pattern = core 2023-12-14T18:49:44.062 INFO:teuthology.orchestra.run.smithi093.stdout:kernel.core_pattern = core 2023-12-14T18:49:44.082 INFO:teuthology.orchestra.run.smithi160.stdout:kernel.core_pattern = core 2023-12-14T18:49:44.118 DEBUG:teuthology.orchestra.run.smithi093:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:49:44.135 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:49:44.135 DEBUG:teuthology.orchestra.run.smithi097:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:49:44.152 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:49:44.153 DEBUG:teuthology.orchestra.run.smithi160:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-14T18:49:44.174 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-14T18:49:44.175 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-14T18:49:44.186 INFO:teuthology.task.internal:Transferring archived files... 2023-12-14T18:49:44.187 DEBUG:teuthology.misc:Transferring archived files from smithi093:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/remote/smithi093 2023-12-14T18:49:44.188 DEBUG:teuthology.orchestra.run.smithi093:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-14T18:49:44.230 DEBUG:teuthology.misc:Transferring archived files from smithi097:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/remote/smithi097 2023-12-14T18:49:44.230 DEBUG:teuthology.orchestra.run.smithi097:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-14T18:49:44.269 DEBUG:teuthology.misc:Transferring archived files from smithi160:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435/remote/smithi160 2023-12-14T18:49:44.269 DEBUG:teuthology.orchestra.run.smithi160:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-14T18:49:44.308 INFO:teuthology.task.internal:Removing archive directory... 2023-12-14T18:49:44.308 DEBUG:teuthology.orchestra.run.smithi093:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-14T18:49:44.311 DEBUG:teuthology.orchestra.run.smithi097:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-14T18:49:44.314 DEBUG:teuthology.orchestra.run.smithi160:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-14T18:49:44.358 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-14T18:49:44.371 INFO:teuthology.task.internal:Not uploading archives. 2023-12-14T18:49:44.371 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-14T18:49:44.409 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-14T18:49:44.409 DEBUG:teuthology.orchestra.run.smithi093:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-14T18:49:44.412 DEBUG:teuthology.orchestra.run.smithi097:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-14T18:49:44.414 DEBUG:teuthology.orchestra.run.smithi160:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-14T18:49:44.426 INFO:teuthology.orchestra.run.smithi093.stdout: 262154 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 14 18:49 /home/ubuntu/cephtest 2023-12-14T18:49:44.429 INFO:teuthology.orchestra.run.smithi097.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 14 18:49 /home/ubuntu/cephtest 2023-12-14T18:49:44.431 INFO:teuthology.orchestra.run.smithi160.stdout: 262153 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 14 18:49 /home/ubuntu/cephtest 2023-12-14T18:49:44.432 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-14T18:49:44.444 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-14T18:49:44.475 INFO:teuthology.nuke:Checking targets against current locks 2023-12-14T18:49:44.754 DEBUG:teuthology.nuke:shortname: smithi093 2023-12-14T18:49:44.755 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-14T18:49:44.777 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi093.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:32.467048', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBK0OZ7Zu/Fbt8VVGDDt1T2PvUhU4fESkuZbiUfSgTrPk3/rXRSJoWSk61bsJ8gYyqxF4U5YRHRR4zvtYnG87JB0='} 2023-12-14T18:49:44.781 DEBUG:teuthology.nuke:shortname: smithi097 2023-12-14T18:49:44.781 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-14T18:49:44.806 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi097.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:32.470154', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBInchYnEXB0mnlv+DG8qY+0DrB4YSZgRo8hrpqJg86p8xIVlwQ42sEF39oDtpwBjjvB0oejWS6OZRCTPycQcfN8='} 2023-12-14T18:49:44.807 DEBUG:teuthology.nuke:shortname: smithi160 2023-12-14T18:49:44.807 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-14T18:49:44.826 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi160.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/choffman-2023-12-14_17:59:30-fs-wip-choffman-62793-distro-default-smithi/7491435', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-14 18:00:32.468462', 'locked_by': 'scheduled_choffman@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNa3qnlFw6f2pdh2OPpaLbJoAkPVdlYJBUYmURBR4LnG+TLB5eIVTAAVAIbzVakwuVbMUo9U2cKgGdoRuPGp2AY='} 2023-12-14T18:49:44.836 INFO:teuthology.orchestra.console.smithi093:Power off 2023-12-14T18:49:44.837 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-14T18:49:44.860 DEBUG:teuthology.orchestra.console.smithi093:power off output: Chassis Power Control: Down/Off 2023-12-14T18:49:44.860 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:44.879 INFO:teuthology.orchestra.console.smithi097:Power off 2023-12-14T18:49:44.879 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-14T18:49:44.887 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on 2023-12-14T18:49:44.899 DEBUG:teuthology.orchestra.console.smithi097:power off output: Chassis Power Control: Down/Off 2023-12-14T18:49:44.900 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:44.912 INFO:teuthology.orchestra.console.smithi160:Power off 2023-12-14T18:49:44.912 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-14T18:49:44.928 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-12-14T18:49:44.932 DEBUG:teuthology.orchestra.console.smithi160:power off output: Chassis Power Control: Down/Off 2023-12-14T18:49:44.932 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:44.952 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2023-12-14T18:49:48.889 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:48.929 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:48.954 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:49.001 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on 2023-12-14T18:49:49.039 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is on 2023-12-14T18:49:49.062 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is on 2023-12-14T18:49:53.006 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:53.040 DEBUG:teuthology.orchestra.console.smithi097:pexpect command: ipmitool -H smithi097.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:53.063 DEBUG:teuthology.orchestra.console.smithi160:pexpect command: ipmitool -H smithi160.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:53.116 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is on 2023-12-14T18:49:53.149 DEBUG:teuthology.orchestra.console.smithi097:check power output: Chassis Power is off 2023-12-14T18:49:53.174 DEBUG:teuthology.orchestra.console.smithi160:check power output: Chassis Power is off 2023-12-14T18:49:53.250 INFO:teuthology.orchestra.console.smithi097:Power off completed 2023-12-14T18:49:53.275 INFO:teuthology.orchestra.console.smithi160:Power off completed 2023-12-14T18:49:57.121 DEBUG:teuthology.orchestra.console.smithi093:pexpect command: ipmitool -H smithi093.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-14T18:49:57.231 DEBUG:teuthology.orchestra.console.smithi093:check power output: Chassis Power is off 2023-12-14T18:49:57.332 INFO:teuthology.orchestra.console.smithi093:Power off completed 2023-12-14T18:49:57.440 INFO:teuthology.run:Summary data: description: fs/workload/{0-rhel_8 begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{k-stock rhel_8} ms-die-on-skipped}} ms_mode/crc wsync/yes} objectstore-ec/bluestore-comp-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts session_timeout} ranks/multi/{export-check n/3 replication/always} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-workunit/fs/misc}} duration: 2570.675473213196 failure_reason: 'Command failed (workunit test fs/misc/layout_vxattrs.sh) on smithi093 with status 1: ''mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bf026da9d69000f3d4a9d16360156095d6612a2d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/misc/layout_vxattrs.sh''' flavor: default owner: scheduled_choffman@teuthology sentry_event: https://sentry.ceph.com/organizations/ceph/?query=1e595971b5a149b2a9beaa6ee6faf175 status: fail success: false 2023-12-14T18:49:57.440 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-14T18:49:57.531 INFO:teuthology.run:FAIL